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W_E. HARRIS 


Looking at Ourselves 
The President's Advisory 
Committee on Campus Reviews 


AN INVITATION 
HAD BEEN LOOKING FORWARD to retirement at the end of the 
| sn: of 1980. However, in April, President Myer Horowitz asked if I 
would chair a steering committee for a couple of years to help start the 
review process for the University. Before I could agree, I wanted to learn 
nore about what a review would entail, and I wondered if I had the 
ippropriate background. 

I studied “A Report to General Faculties Council on Proposed Terms of 
teference for a System of Reviews of Academic Programs,” which was the 
bint report of the Dean’s Council and the Academic Development 
Jommittee. After a couple of weeks of further background reading and 
hought, I accepted the invitation to chair the committee. 
| By June 1980, the steering committee for the reviews had been formed: 
'.E. Harris (myself), Chairman; G. Baldwin (VP Academic, the President’s 
spresentative during startup); J. Forster (Dean of Graduate Studies); 
; ord, and Ms. Florence Watters, Committee Coordinator. 


MEDIATE OBJECTIVES 

The steering committee needed a clear understanding of the purpose of 
‘review. It had to develop sound procedures quickly and establish a 
Vourable climate for reviews on the campus. Inevitably, we would also 
ve to expect to make mistakes and to learn from them. The three main 
iectives were: 
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1. The reviews should have positive goals. In a Folto article, President 
Horowitz had made it clear that the reviews were not to be a policing 
action. They were to help us help each other and would involve 
scholars from within the University as well as from outside, | 
welcomed this kind of approach, having observed it first hand while 
serving as an external reviewer for the Chemistry Department of the 
University of Saskatchewan. 

2. Weshould take advantage of the experience of other universities and 
adapt the best of their procedures. In that same article, President 
Horowitz stated, “We are absolutely determined not to repeat the 
mistakes which have occurred in other institutions.” For example, a 
review system a couple of years earlier at the University of Calgary 
had collapsed. 

3. We must quickly gain the support of the University community, who 


were understandably wary. An effective review would be impossible _| 


without the cooperation and acceptance of the academic staff. 


GETTING STARTED 

On July 9, the steering committee of the President’s Advisory 
Committee on Campus Reviews (PACCR) held its first meeting. We set out 
to obtain information from other universities about their experience with 
reviews. Accordingly, we arranged for a day-long session in September with 
Dr. Blaine Holmlund, the long-time director of reviews at the University of 
Saskatchewan. He gave us a wealth of advice. 

As soon as possible, I spent three days at the University of Saskatchewal 
to obtain first-hand comments from those who had gone through thei! 
successful review process. For more background, I attended a conferenct 
at the University of Guelph. F. Watters and I attended a major conferencé 
on academic reviews sponsored by the University of Illinois. 

The PACCR committee met with Dr. R. Miller, a consultant who had 
written several books on the general subject of evaluation. Dr. Rick Harpe! 
of the University of Colorado visited us and told us about his experienc: 
He said that they had initiated nine departmental reviews in one year and 


nine more in the next, and that they now needed a year off just to catch UP 
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We were advised to start the review process slowly, with small, well-run 
iits so that we could develop our procedures under favourable conditions. 
ter that, we were strongly advised to limit reviews to about ten units in 
y one year, and certainly never more than fifteen. 

Because of the diversity among departments and faculties across 
mpus, the Joint Report of the Dean’s Council and the Academic 
velopment Committee recommended that the procedure should be 
sible. It would be a mistake to develop detailed standard operating 
iedures that would presume to fit all units. Rather, we encouraged each 
yartment and each review team to tell its story in its own way. Broad 
eral instructions rather than detailed directives were appropriate for 

units and for the reviewers. 

Ve attempted to keep the university community informed about our 
vities and philosophy from the beginning. At the end of October, the 
isident’s Advisory Committee of Chairmen invited me to meet with their 
ip and talk about the objectives of the reviews. 

Ithough we were inexperienced, it was important to talk to this group 
in early stage. I described in detail the attitude and the common-sense 
joach to the review process we intended to use. Although most 
hibers of the Chairmen’s Council simply listened, Dr. T. Nelson and 
iG. Marahens appeared determined to stop the review process. VP 

jwin and President Horowitz addressed their concerns as best they 
jd and assured them that the reviews would proceed. 
hat meeting proved crucial. During the weeks that followed, several 
"men quietly approached me and volunteered their departments for 

w during our second year. These included Dr. G. Prideaux of 

listics and Dr. T. Burton of Recreation Administration. Their 

‘eration was extremely important for the development of our 
}dures, 

velcomed the opportunity to talk with anyone interested in our 
isophy since it helped to clarify my thinking. I gave an invited talk to 
jaculty of Education in the auditorium with about 250 present. | 
ljmed the request of the capable Kdmonton Journal reporter, Ron 
Aiers, for an interview. After a few minutes, it became clear to him 
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srnative, and so with a sense of relief and regret, I submitted a letter 
ignation to Dr. Horowitz. A couple of hours later I received an 
ency call from VP Baldwin: “Dr. Horowitz and I have to leave for 
ridge today. Don’t do anything!” Two days later he and the President 
me in and told me, “We refuse to accept your resignation. Dr. Ford 
epped down [from the PACCR committee].” 


2>ACCR COMMITTEE 
gy oe VP Baldwin asked Dr. Fred Enns to 
talk to me concerning the possibility 
of filling the vacancy in the PACCR 
steering committee. When we met, he 
immediately remarked that he 
_ thought a flexible, humanistic review 
process should be developed. I 
responded with a “hooray.” Fred Enns 
; told me later that he had been wary at 
our first meeting. 

As the months progressed, it was an 
ongoing pleasure to work with him as 
an intelligent, positive, constructive 
colleague for the remainder of all of 
views. Dr. Forster became increasingly ill. After he died, President 
itz asked former academic VP Dr. Kreisel to join the steering 
ittee. Dr. Kreisel was a wise, congenial colleague to help “light the 
ff positive reviews during the years that followed. When the non- 
mic units were included in the review system, W. H. Jopling was 
to the PACCR committee. Bill Jopling was a perfect choice to handle 
of our initial interactions with non-academic units. 

Single committee to oversee the reviews of academic and non- 
mic units worked well. During the remaining decade, the four of us 
, Harris, Kreisel, Jopling) had well over one thousand amicable, 
ight productive meetings. 

the spring of 1981, the PACCR Coordinator, Florence Watters, was 


CCR steering committee 
ter Harris, Chairman, Henry 
, Fred Enns, Bill Jopling 
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transferred to another position. Dee Budd in the President’s office filled in 
for the next few months, but the additional workload became excessive, In 
December 1981, Lorrie Pearson was appointed as full-time coordinator and 
was assisted by Lu Ziola for a couple of years. The PACCR headquarters was 
moved to the Chemistry Building, physically and symbolically separated 
from central administration. When Lorrie Pearson retired, Shirley Moore 
was appointed to the position of Coordinator. | 


In 1980, I agreed to undertake a two-year commitment as Chairman. As 
time went on, President Horowitz asked me to extend my commitment | 


several times. Finally our arrangement became open-ended, with either of 
us able to terminate my involvement with six month’s notice. Near the end, 
when dealing with some large and some difficult units, it seemed 
inappropriate to turn the direction over to a new steering committee, and 
I continued until all of the 127 units had been reviewed during eleven 
years. 


DEVELOPMENTS AND COMMUNICATION 
In the second year of the PACCR review process, we gained more 
experience and refined our procedures under a full load of fifteen units. We 


became involved with units from all major faculties as well as non | 
academic departments. To start a review of a unit, PACCR met with the | 


members of the unit. After about a fifteen-minute presentation, we invited 


questions and comments. For a unit self-study, we made available some | 


examples of earlier self-studies. We offered any other help that we could 


reasonably give. Dr. Ted Holdaway of Institutional Research and Planning 


provided academic departments with basic data for use in their sel 
studies. ¥ 

A PACCR review of a unit entailed three major reports: the unit’s Sell 
Study, the report of the Unit Review Committee (URC) and the Unit 
Response. The URC normally consisted of four members. The unit was 


asked to provide nominations for two members from outside the University 
but from the discipline and nominations for someone from a related unit 
within the University. PACCR nominated the university-at-large member i 


act as a surrogate for the interests of the wider university. 
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» members of PACCR prepared a summary of the three reports for the 
lent. Before we met with the President, we met with the Head of the 
yr an oral exchange of conclusions and perceptions. We then met with 
resident who then met with the Head of the unit along with the 
priate VP and the Dean. PACCR also wrote a confidential letter to the 
ers of the URC to give them our perception of the results of the 
. It was clear from the responses we received that the members of 
2C had more than a transient interest in the unit they had reviewed. 
’CR met with and reported annually to several groups; the Priorities 
lanning Committee, the Deans’ Council and the Chairs’ Council. We 
let once with a Senate committee. Reports were published in Folzo, 
1 booklet entitled Campus Reviews was made available to 
fments, reviewers and others interested. 

ore each on-site visit by URC, we sent a notice to Folzo inviting input 
ny who wished to provide information or comment to the URC. The 
nts’ Union and the Graduate Students’ Association were also kept 
ied. Students associated with academic departments provided 
uled input directly to the URC. 

sr our seventh year, President Horowitz asked us to review the 
{ process. In our seventh annual report, we made the following 
mendations for consideration and discussion: 

he PACCR process should be repeated and built into the 
dministrative structure of the University. 

he crucial role of the President should be recognized and 
laintained. 

he Advisory Committee for steering the reviews process should be 
1ade up of senior members of the University community who have 
ride experience and sound judgment, but who do not currently hold 
dministrative positions. 

he Deans of academic units and Vice Presidents for 
dministrative/support units should have the main responsibility for 
ionitoring progress towards achieving agreed-upon 
scommendations arising from reviews, with the Vice President 
Academic) or Vice President (Administration) retaining overall 
\onitoring responsibilities. 
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5. The President should have a reasonable sum of money each year to be 


10. 


An example of a review team. This review team was for the microbiology 


used to respond to the findings of the reviews. 

Program improvement should continue to be the major objective of 
reviews. 

The reviews should be carried out as part of a process in which a unit 
examines its current strengths and limitations with a strong forward- 
looking emphasis. 

The reports of Unit Review Committees should continue to be treated 
with confidentiality. 

Single review teams, typically two external and two internal 
reviewers, should continue to be the norm. 

Practices and procedures currently being used should be carefully re- 
examined and adapted for the longer term. 


if 
TSR | 


department. (L to R): R. Colwell (former President, University of Maryland, now 
President of the National Science Foundation), R. Wolfe (University of Illinois), 
B. Wilkinson (Economics Department), L. Milligan (Animal Science Department) 
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Near the end of the reviews of all the units, the PACCR steering 
mmittee wrote a major final report entitled “President’s Advisory 
mmittee on Campus Reviews, Tenth Annual Report.” In one section of 
e Tenth Annual Report, the PACCR process is described in detail, 
cluding the self-study, the time problem, reviewers, URC reports, unit 
sponses and the PACCR process in perspective. In another section, the 
jor issues addressed by reviewers are described. One appendix is the 
ird edition of Campus Reviews. A second appendix lists all the units that 
re reviewed along with the names of the reviewers. 


ISULTS 
The results of the 127 PACCR reviews cannot be given in detail in a page 
two. From my prejudiced point of view, I think every unit received 
nefits—but not always in the economic sense. How much benefit 
pended in part on the attitudes of the people in the units. When there 
is a will to bring about important changes, particularly within the 
idership of a unit, then much could and did happen. The reviews helped 
identify major problems and point the way to their solutions. The URC 
port provided a focus for important discussions and served to encourage 
ange, renewal or reconstruction. The continuing interest of the 
esident contributed much to the success of the review process. 
The reviews encouraged serious and systematic consideration of the 
al work of a unit and acted as a catalyst for program improvement. As a 
sult of a review, many changes took place that would not otherwise have 
curred. Sometimes serious problems that had been allowed to fester 
re resolutely dealt with. Most units saw a review as an opportunity to 
amine the past, evaluate the present and consider directions for the 
ure. 
The following is a condensation of major issues addressed by unit review 
mmittees. 
Departmental renewal. The reviews found that departments may go 
through something akin to a life cycle of initiation, growth, plateau 
and decline. Appropriate action needs to bring about a process of 
renewal before decline sets in. 
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Administration. Unit Review Committees often concluded that 
insufficient attention was paid to the administrative component of q 
unit’s operations. : 
Aging equipment. There are substantial problems and costs relating 
to the upkeep and replacement of equipment and buildings. 
Communication. Many review teams commented on the inadequacy 
of communication as a factor in internal and external problems and 
relations. I believe that studies have shown more broadly that lack of 
communication is a common problem. 

Graduate studies. Reviewers indicated that often the time taken for 
students to complete their studies was too long. Not all departments 
clearly enunciate admission procedures, and they often fail to 
monitor the progress of their students. 

Transition to research and teaching. The modern university places 


increasing emphasis on scholarship and research. The transition from 


a mainly teaching institution may require reorientation of staff. 

Teaching workloads. Especially during the first three or four years of 
the reviews of academic departments, a case was sometimes made for 
more resources on the basis of a heavy teaching load. It seemed not 
to matter whether the teaching load was really heavy or not. For 
example, one zealous chairman of a department that had a teaching 
load that was almost certainly lighter than all but two in the 
University was able to persuade his Dean to assign more resources 
and obtained a teaching load lighter than all but a single department 
in the whole University. Departmental instructional duties are diverse 
and an indicator of teaching load must include lecture hours, number 
of students, laboratory instruction and teaching assistants. Are 
teaching loads directly proportional to the number of lecture hours? 
No. Are they directly proportional to the number of students taught? 
No. PACCR had to come to grips with the issue of teaching loads and 
encourage departments to use valid numerical comparisons of loads 
as an aid to common sense. Arguments based on teaching loads were 
used in a less manipulative way in the last few years of the reviews. A 


validated indicator of teaching load is described in the Tenth Annual | | 
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Report beginning on page 41. 

Teaching and research. The workload for academic personnel 
includes undergraduate teaching, research, _ professional 
responsibilities and administrative duties. On the matter of teaching 
and research, teaching can be a source of research ideas and research 
a source of teaching content. However, a common perception is that 
undergraduate teaching duties inhibit research activities. Another 
perception is that high competence in research endows one with high 
competence in the teaching function. PACCR studied the question of 
teaching and research and found that the correlation coefficient of 
the relation of research productivity to teaching competence is 
neither negative nor positive and is actually zero. 

Diversity. Early in the review process, I concluded that there is a 
sorting mechanism in society that somehow causes individuals to 
gather into groups with unique and different characteristics from 
other groups. Once formed, the members of the group tend. to 
understand and support each other. For example, consider groups of 
chemists, lawyers, pathologists, accountants or surgeons. Each group 
differs from each of the others—not that they are better or worse, but 
different. On a broader scale, those in the arts units differ from those 
in sciences. Review teams for Arts units needed more time for 
discussion than did units in Science. Contrary to what I first had 
assumed, I think that those in Arts strive to attain and have more 
faith in numbers than those in Science, who approach numbers with 
a skeptical attitude. 


MENTARY 
dditional material with respect to the PACCR review process is 
lable in the University Archives as follows: 
The draft of this article (12 pages) 
The Tenth Annual Report of PACCR (58 pages) 
The President’s Advisory Committee on Campus Reviews, Annual 
Reports, 1981-1990 (800 pages approx.) 
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An article entitled “Reviews: Catalysts for Improvement” in the | { 
President’s Club Newsletter, May 1988, described the PACCR review y 
process and stated: | 

This [review] process brings together the best minds to concentrate 
on recommendations for each department or administrative unit, 
In addition, it tends to be a positive force in creating awareness of 
our University beyond the urban, provincial, and national yrt 
boundaries. 


One example of program improvement involves the Registrars 
Office. The Unit Review Committee identified problems with 
students’ registration. Mr. Brian Silzer, who had just been 
appointed registrar, used the analysis and recommendations as 
the basis for the development of telephone registration. 

The success of the PACCR review process was recognized by other 
universities contemplating campus-wide review programs. Requests were 
received to describe and make recommendations about initiation of review 
programs in other institutions. They include University College of Cape 
Breton, University of Montreal, Guelph University, University of Calgary, _ 
Agriculture Canada and an Arts and Science Deans meeting in Victoria. 

The Tenth Annual Report was circulated to many interested persons. 
The report prompted a number of unsolicited comments including the 
following: 

There is no doubt in my mind that your personal integrity, coupled 
with your patience and perseverance in emphasizing a positive 
approach to program improvement, were the key factors in the 
acceptance by the University community. A 
Florence Watters (PACCR Coordinator during the first year) 


Your contribution to the University has been enormous. You have : 
brought wisdom, judgment and the voice of reason to what im all 
probability has been the single most important review the : 
University of Alberta has had or will ever see. : 
Peter Meekison (former VP Academic) 
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‘We have just entered the fourth year of our program. In general, we 
are still following your procedures with afew changes to fit into the 
Guelph scheme of things. 

Stanley (University of Guelph, Internal Review Committees 
Coordinator) 


_ Dr. Fred Enns kindly read and reviewed the more extensive draft of the 
| PACCR millennium project report and gave valued advice. Dr. Enns gave 
| me permission to include the following personal comment. 

: I enjoyed all my work at the University of Alberta over the years, 
but I count the time working on PACCR as the most fulfilling and 
satisfying. To work with colleagues like Walter Harris, Henry 
Kreisel and Bill Jopling was a rare privilege. Because the reviews 
got us into all parts of the University, both academic and non- 
academic, I gained an appreciation of the University’s stature and 
made me proud to be a part of it. The opportunity to meet eminent, 
world-renowned authorities from all the disciplines, who formed 
our Unit Review Committees, is afforded to few people. I am 
grateful to have been one of those few. And the feeling that we were 
making a significant contribution to the University of Alberta was 
like icing on the cake. 
| There is not the slightest doubt that a successful review process could 
{not have been possible without the contributions of many individuals. I do 
re attempt to name them all. 

_ It was an enormous pleasure to work closely on a day-to-day basis with 
red Enns, Henry Kreisel, Bill Jopling, Florence Watters, Lorrie Pearson, 
la Ziola and Shirley Moore. The continuing interest and contributions of 
‘Dr Ted Holdaway are appreciated. The advice of Dr. Blaine Holmlund at 
the beginning was crucial to the startup. The interest and dedication of 
President Horowitz was of decisive importance. It was a delight to get to 
jknow and work with numerous others including Vice Presidents, Deans, 
|the Heads of all the units in the University and with students. 

The cooperation of 250 of my University of Alberta colleagues was heart 
Marming They accepted my invitation to act as reviewers and gave loyal 
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service (with no honourarium) to units under review and to the University, 


It was stimulating to meet and work with 290 scholars who were our 


reviewers from outside the University in the diverse fields of learning. The 


cumulative advice received from this group of more than 500 scholars has _ 
been priceless for the current welfare of the University. Their names are 


listed in the Tenth Annual report. 

To end on a personal note, I have been privileged to have had three 
careers at the University of Alberta during forty-four years. First was a 
decade as a teacher of about 500 students per year. Second was two-and-a- 
half decades with a career in teaching, research administration and 


professional responsibilities. Third was the decade of involvement in the 


reviews of all the units in the University. Each of the three careers had its 
special rewards and enriched my life. 
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INTRODUCTION 


All the on-site reviews for both academic and administrative/support units, 
carried out under the guidance of the current PACCR committee, will be finished 
by the end of 1990. At the time of writing (end of June 1990), there are six units 
(Computing Science, University Computing Systems, Dental Health Care, Finance 
and Management Science, Library System, and University Health Services) yet to 
have on-site visits by unit review committees. After their visits, their reports will 
have to be dealt with. The responses of these units, along with the responses of a 
small number of other units, will also have to be processed. This, the Tenth 
Annual PACCR Report is therefore the last to be submitted by the current steering 
committee. 


Much has happened since the PACCR process was established 10 years ago. 
The period has not been an easy one for the University. Major adjustments have 
had to be made, and many of the recommendations of Unit Review Committees, 
particularly those that involved financial commitments, could not be acted upon. 
But many recommendations could be, and were, implemented, and helped to 
achieve the major goal of the process, that of program improvement. 


The PACCR process became widely known throughout Canada, and many 
universities consulted us when they were establishing their own review 
procedures. Dr. E. A. Holdaway, of our own Department of Educational 
Administration, has had a continuing interest in the PACCR process and has been 
instrumental in the publication of articles on it. 


1) Holdaway, E. A. Ten Years of Program Reviews at the University of Alberta. 
Paper presented at the Annual Conference, Canadian Society for the Study of 
Higher Education, Victoria B.C. (June 1990). 


2) Enns, F., Harris, W. E., Holdaway, E. A., Jopling, W. H., and Kreisel, H. 
Program review: The experience of a Canadian university. Higher Education 
Research and Development (Higher Education Research & Development Society of 
Australasia) 5, 31-47 (1986). 


3) Harris, W. E., & Holdaway, E. A. Systematic reviews of university programs 
and units. Canadian Journal of Higher Education, 13, 55-76 (1983). . 


4) Holdaway, E. A. The review process in universities. Australian Journal of 
Higher Education, 15, 106-120 (1981). 


Over the years, the annual PACCR reports have attempted to inform the 
university community about the PACCR process itself and the way in which it 
evolved and changed; of more importance, the annual reports have attempted to 
draw general conclusions from the recommendations of Unit Review Committees. 
Very often what seemed to be problems faced by specific units had wider 
ramifications and were in fact problems faced by departments across the 
University. 


We thought it would be useful to present, in this, the last annual report of the 
current steering committee, abstracts that would bring together the main issues 
that were addressed by the 126 Unit Review Committees, as well as to present a 
summary of the PACCR process itself. Each abstract gives the number of the 
annual report in which it appears, together with page references, so that someone 
wanting further elaboration of a particular item can find it. 


Unit Review Committees have made an inestimable contribution to this 
University. They have uniformly made thoughtful and serious recommendations 
designed to assist units in the goal of program improvement. Many units were 
disappointed when new resources could not be made available. We should say, 
however, that when there was clearly a pressing need for a modest injection of 
new resources, the President could, and did, act to meet at least part of the need. 


On behalf of the University, the members of PACCR wish to express 
appreciation to our own colleagues for the major contribution they have made 
when they served on Unit Review Committees. With but few exceptions, they 
have willingly agreed to serve when asked, and they have helped in numerous 
ways to catalyze important changes during the decade of the 1980s. In addition 
appreciation is expressed for the vital and essential role placed by the PACCR 
Coordinators and most particularly, Mrs. Shirley Moore during the last four years. 


ZG 


H. Kreisel 


THE PACCR PROCESS 


A) THE SELF STUDY (6th, p. 1) 


B) 


C) 


The task of preparation of a self study is often perceived to be onerous, 
although at the same time many departments have commented that the self 
study was ultimately the most important and rewarding part of the process. 
Reviewers have commented on the “unique opportunity” the self study offers 
departments “to set aside the precious time necessary for planning and consensus 
building. . . . These periods of intense self-scrutiny often help to develop new 
directions for the unit, better understanding of what others are doing, and a stronger 
group identity.” 


LENGTH OF REPORTS INVOLVED IN THE PROCESS (4th, 
pp. 7-8) 


Reports have varied considerably in length. For Self-Study Reports the 
median length is 74 pages, for Unit Review Committee Reports 17 pages and 
for Unit Responses 6 pages. Corresponding means are 88, 19, and 11 pages. 


COMMENTS ON THE TIME PROBLEM (5th, p. 14) 


a) Time to Set Up a Review 


We have found that, typically, it took 3 to 4 months from the time of 
nominations for the Unit Review Committee to the on-site visit. This 
meant that a self study had to be completed and a Unit Review 
Committee selected before Christmas if there was to be much hope for 
scheduling the on-site review before the end of classes in the spring. If 
the on-site activity was to take place in the fall, then the self study and 
selection of the review committee had to be completed during the 


spring. 


b) Duration of Visits of Unit Review Committees 


Most review teams were on site for 3 days. In the beginning, 2-day 
reviews were common, but it became evident that 2 days was too short 
a time, even in the case of relatively small departments. Thus 2-day 
reviews were eliminated in favor of 3- or 4-day reviews. A review team 
needs adequate time to conduct the necessary interviews and time to 
itself to reach conclusions and formulate recommendations. Overall we 
scheduled 26 reviews of 2 days, 82 of 3 days, and 18 of 4 days. 


D) PACCR REVIEWERS 


PACCR used as reviewers persons from Canada or continental United 
States, ranging from Victoria, B. C., to St. John’s, Newfoundland, and from 
Alaska to Georgia. 


PART I 


PARTS 


In Canada our 177 reviewers came from 26 different universities and 13 outside 
agencies in 9 provinces. The two universities that provided the most reviewers 
were the universities of British Columbia and Toronto. From the United States our 
66 reviewers came from 37 universities and 3 outside agencies in 23 states. 


Within the University our reviewers came from all sectors, with the largest 
number from the Faculty of Arts. With few exceptions, we did not ask an 
individual to act as a reviewer on more than one occasion. In more detail, the 
distribution was as shown in Appendix B. 


E) COMPOSITION OF REVIEW TEAMS (3rd, p. 3) 


Review teams were normally composed of four members. Two members came 
from outside the University, but from the discipline that was being reviewed. Two 
came from our University: one from a related unit and one from the 
University-at-large. This mix of reviewers turned out to be most effective. 


Review teams recognized, to quote from one of the Unit Review Committee 
reports, that they could not, “in the space of a few days, expose fairly and fully all the 
strengths and weaknesses of an active . . . teaching and research department. Inevitably, 
too, what preoccupies a review committee are not the achievements and distinctions of a ~ 
department but its problems and shortcomings, so that reports such as this tend to be critical 
documents. We describe (the Department's problems) frankly and attempt to offer 
constructive recommendations for their redress.” 


F) REPORTS OF UNIT REVIEW COMMITTEES (5th, p. 9) 


Unit Review Committees were often outspoken in their comments, which 
sometimes seemed harsh. Departments were then likely to react strongly, 
sometimes negatively. 


It is important to emphasize that the report of the Unit Review Committee, 
important though it was, was only one of three documents that constitute the full 
documentation of the review process: the Self Study, the Unit Review Committee 
Report, and the Departmental Response. 


Departments did not need to accept every recommendation of the Unit Review 
Committee. It is, however, necessary that comments by respected individuals, both 
from within and from without the University, be seriously considered. 


G) RESPONSE TO REVIEWS (6th, p. 3) 


Since Unit Review Committees often stressed problems that needed attention, 
and the tone was therefore critical, the initial response sometimes tended to be 
defensive. But, when it became clear that the real concern was to help, the initial 
attitude was replaced by a more positive one. With few exceptions, units 
eventually gave serious attention to explicit recommendations. 


THE;PACCK PROCESS 


H) RANK ORDERING OF UNITS UNDER REVIEW (5th, p. 15) 


From time to time, it was proposed that PACCR ask review teams to rank the 
unit under review, that is, to indicate how the unit ranked in comparison with 
similar units elsewhere. PACCR consistently rejected the proposal as being 
unworkable and as interfering with other important aspects of the review process. 
A question that persisted in our minds was: “How would ranking contribute to the 
main objective of program improvement?” 


Since the objective of the PACCR process is to bring about program 
improvement, attention should be directed to ways in which exactly such 
constructive changes can be brought about. Hence the examination - in the 
self-study phase, the site visit, and the follow-up - is conducted in a dispassionate, 
nonthreatening way, in which strengths and weaknesses can be exposed and 
discussed and advice on how to overcome problems can be given and considered. 
If a stated - or even implied - purpose were to make comparative rankings, the 
focus of attention would be different throughout the review: the self study, the site 
visit, the report of the Unit Review Committee, the departmental response, and 
the final disposition by the President and the Planning and Priorities Committee. 


I) FOLLOW-UP PROCEDURES 
a) Internal (9th, pp. 12 - 15) 


The review process involved several phases or stages, one of which was 
the follow-up some 18 - 24 months after meetings with the chairman 
and president. The purpose was to assess the extent to which major 
recommendations and agreed-upon actions were actually carried out and 
the degree to which program improvement had occurred. The follow-up 
procedure was lodged with the office of the appropriate vice-president. 
General feed-back indicated that the follow-up has been less successful 
and effective than was expected. This situation may be due to the 
remoteness of people in the offices of vice-presidents, in management 
distance, but also in terms of time available and directness of 
involvement. Hence it has been suggested that at least in 
departmentalized, academic faculties the follow-up be assigned to the 
dean of the faculty. In administrative/support units the process should 
remain with the Vice-President (Administration) as at present, and in 
the case of nondepartmentalized or small faculties it should remain with 
the Vice-President (Academic). 


b) Letters to Reviewers (5th, p. 3) 


In the fifth year of reviews, PACCR began writing a letter to members of 
the Unit Review Committees, giving our perception of the results of the 
review up to the point where PACCR had no more formal activities, that 
is, after we had discussed the review with the President. Some replies to 
these letters are worthwhile quoting: 


It seems to me that the general problems of teaching load and teaching vs 
research are universally difficult issues. 


For my part, I enjoyed my participation in your review process. I found it 
intriguing and I commend your university for having the self confidence 


PARTY 


necessary for such a process of self examination. Would that others might follow your 
example. 


I have taken part in a number of such reviews at other universities, but this is the 
first time in my experience that anyone has taken the trouble to respond in this way. I 
want to let you know how much I appreciate your courtesy. I was also very pleased to 
hear of the progress which the Department is making. 


Most often, after a review process of this sort, members of external review teams 
are never informed about institutional reactions . . . your process is thorough and 
qualitatively sound. As you know, I have been involved in many external reviews but 
I have yet to find a process so well conceived and pursued. 


J) PACCR REVIEWS AND ACCREDITATION (9th, pp. 4 - 7) 


The question of overlap between accreditation and PACCR reviews was often 
raised by departments offering professional programs. Although some overlap 
exists, the two types of reviews emphasize different aspects of programs. The 
PACCR process is intended to emphasize planning through examination of such 
matters as staffing needs and options; undergraduate, honors, and graduate 
programs; teaching and research productivity; stimulation and renewal of 
scholarship; and development of new areas of specialization. The self-study 
process and report provide opportunity for assessment of the current status and of 
projections for the future. In a comparison of the PACCR process with other 
reviews, a distinguished member of the Faculty of Medicine who has extensive 
experience with both types of reviews listed eight benefits of PACCR reviews: 

1. Detailed and comprehensive. 

2. Forces careful self-examination. 


3. Freedom of reviewers to interview all who may have relevant information to 
MA 
give or observations to make. 


. Careful selection of the review team. 

. Need for reviewers to be professionally responsible. 
. Publicity value to good departments. 

. Stimulative effect for departmental administration. 


. The existence of the process keeps departments academically and 
professionally on their toes. 
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K) PACCR PROCESS IN PERSPECTIVE (7th, pp. 2 - 12) 


In the Seventh Annual Report, PACCR set the whole process in perspective, in 
order to arrive at some conclusions regarding the whole experience of reviews. We 
concluded that many important changes have been brought about as a result of the 
review process, although it was clearly not possible to produce a simple balance 
sheet. 


The fact that reviews were carried cut not only for academic units but also for 
administrative/support units was important, and PACCR concluded that the 
decision to include the latter was wise. 


inEeerACGR PROCESS 


PACCR reviews must be seen as part of a long-range process: Unit Review 
Committee reports can play a major role in program improvement. 


Examples from both academic and administrative/support units are included to 
show what can happen when a unit fully enters into the process set in motion by a 
Unit Review Committee report. Clearly without the stimulus given by Unit Review 
Committee reports, comprehensive discussions about the nature of necessary 
changes would not likely have taken place. PACCR reviews have brought 
important matters to a higher level of visibility and forced examination of matters 
that had previously been ignored. Some of the problems that need to be addressed 
will take a long time to be worked out. 


Examples are given of recommendations by Unit Review Committees that 
involve profound and sometimes radical changes in the work of a department (pp. 
8-9): 


Unit Review Committees have made suggestions as to the need for existing 
promotion policies to meet higher standards; improvements of scholarly 
productivity; the replacement of retiring staff members so as to bring about both a 
revitalization of a department and a reordering of scientific and scholarly priorities; 
what a search committee should look for when new staff members are to be 
appointed; ways of improving communications; radical re-examinations of the 
undergraduate, and particularly the graduate, curriculum; future developments in 
a department’s work designed to make the department a leader in its field; 
elimination of duplication of effort. These examples are by no means exhaustive, 
but give an idea of the wide-ranging nature of Unit Review Committee 
recommendations. 


An attempt was made to give a collective profile of the University (pp. 9-10). We 
can say that the University of Alberta is a first-rate university, particularly in its 
undergraduate programs. The reputation of the University as a graduate and 
research institution is more mixed. A few departments, spread through all the 
major faculties, have achieved truly international standing. Just below this group 
are a number of departments on the threshold of achieving international standing. 
A still larger group has a national reputation. The others must be rated as 
important on the provincial scene, but have yet to create national or international 
reputations. 


The last part of the Seventh Annual Report (pp. 13-20) addresses the future of 
review activities at the University, and gives background and rationale for 10 
recommendations: 

1) That the PACCR process be repeated and built into the administrative 
structure of the University. 


2) That the central role of the President be recognized and maintained. 


3) That the Advisory Committee for steering the review process be made 
up of senior members of the University community who have wide 
experience and sound judgment, but who do not hold administrative 
positions. 

4) That the deans for academic units and associate vice-presidents for 
administrative/support units have the main responsibility for monitoring 
progress towards achieving agreed-upon recommendations arising from 
reviews, with the Vice President (Academic) or Vice President 
(Administration) retaining overall monitoring responsibilities. 

5) That the President have a reasonable sum of money each year to be 
used to respond to the findings of reviews. 
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6) That program improvement continue to be the major objective of 
reviews. 

7) That the reviews be carried out as part of a process in which a unit 
examines its current strengths and limitations, with a strong forward 
looking emphasis. 

8) That the reports of Unit Review Committees continue to be treated with 
considerable confidentiality. 

9) That single review teams, typically involving two external and two 
internal reviewers, continue to be the norm. 

10) That practices and procedures currently being used be carefully 
reexamined and adapted for the longer-term future. 


As a result of this section, a committee chaired by Professor Robert J. Crawford 
was set up to review the PACCR process and make recommendations. The report 
of the Crawford committee differed in some respects from the recommendations 
cited here. It was widely discussed during the 1989-90 academic year, but has not 
yet June 1990) been acted upon. 
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MAJOR ISSUES ADDRESSED BY UNIT REVIEW 
COMMITTEES 


DEVELOPMENT OF ACADEMIC DEPARTMENTS (8th, p. 10) 


and often illustrious period of competent teaching, scholarship, and research, 
have apparently reached a plateau. Although work is still of high quality, the 


| Some Review Committees have identified departments that, after a lengthy 
: unit has lost some of its earlier sharpness. 


It appears that departments go though something akin to a life cycle. The 
| cycle includes initiation, a period of struggle to become established, a period 
of growth, a plateau period where things go well without noticeable growth, 
: a period of decline through loss of staff or morale and displacement of goals. 
With careful monitoring and early detection of symptoms, appropriate action 
can be taken to bring about a process of renewal. 


ACADEMIC UNITS - STRUCTURE AND ADMINISTRATION 
| (8th, p. 8) 


| Review Committees have often commented that frequently insufficient 
attention is paid to the administrative component of the unit’s operation. 
Compared with teaching, scholarly work, and research, administration may 
| be considered unworthy of serious consideration. When this is so, 
administrative functions are neglected or distributed haphazardly, with the 
result that the academic activities themselves suffer from lack of facilitative 
support. 


ia A common criticism by Review Committees was overinvolvement of staff 
in committees, and the recommendation was that the number of committees 
be reduced by combining the functions of several. 


Although giving adequate attention to administrative functions is a 
concern, Review Committees do not look favourably on the appointment of 
co-chairs in any but the largest departments. 


Among administrative functions that must be done by academics is the 
program counselling of students. Most Review Committees do not approve of 
assigning this function to administrative officers. 


ADMINISTRATIVE/SUPPORT UNITS (4th, p. 6) 


Our experience has shown that the appointment of one committee to 
oversee the reviews of both academic units and administrative/support units 
worked very well. It is a better approach than having different committees for 
each area. The continuity that one committee provides, the base of 
knowledge resting in one committee, rather than two, and the fact that some 
issues cross from academic to administrative/support units and vice versa 
demonstrate that one properly structured steering committee can do the 
necessary work expeditiously and efficiently. 
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The first part of this report, which deals with the PACCR process, is as 
applicable to administrative/support units as it is to academic units. However, 
the second part of this report deals almost entirely with academic 
matters—exceptions are communications, aging equipment, and use of 
resources. Because each administrative/support unit is unique in what it does, 
there are few common issues such as are found in academic units, which are 
involved in the similar activities of undergraduate teaching, graduate 
teaching, research, and scholarly activity. 


PROBLEM OF AGING EQUIPMENT (9th, p. 10 - 11) 


A university is expected to be at the forefront of research and the 
advancement of knowledge. Hence, scientific and technical equipment must 
be up-to-date and in working condition to educate scientists, engineers, and 
technologists for the immediate future. Keeping equipment in repair and 
replacing obsolete items is expensive. When, as with major laboratory 
facilities, the building and its systems is an integral part of the equipment, 
there are substantial problems and costs of upkeep and replacement. 
Computers and computer-related equipment have a notoriously short —_- 
life-span; unless replacements are continuously made, students are in danger 
of learning out-dated procedures no longer relevant in business and industry. 


CAMPUS REVIEWS, Third Edition 


Campus Reviews is a booklet prepared as background information for 
members of the university community and for our reviewers. It includes a 
section on our philosophical approach to reviews and another on the review 
process itself. These two topics were the focus of much discussion in our first 
2 years of operation. The other two major sections deal with academic 
workloads and the PACCR process in perspective. Campus Reviews (without 
appendices) is included as Appendix A of this report. 


COMMUNICATIONS (8th, p. 13) 
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Because so many Unit Review Committees commented on the need for 
better communication both internal and external, we saw fit to address the 
matter twice, in both the Fifth and Eighth Annual Reports. 


Many review teams made comments to the effect that the inadequacy of 
communication is a factor in many internal and external problems that face 
both academic and administrative/support departments. Several Unit Review 
Committees stated that with improved communications, units could do a 
better job than they are now doing. 


It is easy to agree on the need for extensive, effective communications. It is 
more difficult in the press of normal activities to be aware of the specific areas 
of inadequacy or breakdown. 


Almost without exception the Unit Review Committees have pointed out 
the need for administrative/support units to better acquaint the campus with 
the services they have to offer. The campus community is often unaware of 
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what services particular units have to offer and does not understand why the 
units do certain things. 


The need for better communication is something that both academic and 


administrative/support units should continually address. 


GRADUATE STUDIES (5th, p. 7) 


a) 


b) 


General Comments 


On the basis of the comments made in a number of Committee 
reports about Graduate Studies, PACCR made a report to the Dean of 
Graduate Studies and Research. 


It should be said that in most cases the Unit Review Committees 
found that the level of graduate work in departments was satisfactory 
and often distinguished. In one or two cases the Committee felt that 
departments had overextended themselves and did not have the 
resources to carry on doctoral work in certain areas, and they therefore 
recommended retrenchment. 


More often Unit Review Committees commented on procedures that 
needed to be tightened up or revised. Not all departments had clearly 
enunciated admission procedures, and they often failed to monitor the 
progress of graduate students. In some departments students with 
similar qualifications were treated differently because prescriptions for 
graduate programs were often left up to individual staff members. 
Proper and disciplined supervision of graduate students was also a 
matter of frequent concern. Some departments experienced high 
drop-out rates, which the Committee in question regarded as a matter 
that urgently needed attention. 


In several reports the Unit Review Committees questioned the length 
of time taken for students to complete course requirements and urged 
departments to reconsider their policies (see next section). 


PACCR recognizes that graduate work is a highly individual 
undertaking and that departments must be given the widest degree of 
flexibility to design programs for their students. There cannot therefore 
be tight central control. But, it is the responsibility of the University and 
of the Faculty of Graduate Studies to establish clear guidelines and to 
ensure that departments and individual staff members carry out their 
responsibilities in a conscientious manner. 


Time for Completion of Degrees (9th, p. 8) 


Reviewers have indicated that it often takes too long for students to 
meet graduate degree requirements. This concern is present at most 
universities and will never be wholly resolved. The graduate thesis or 
dissertation should be looked upon as a study introducing the candidate 
to the field of research, rather than as a definitive work on a given 
subject. Directors of research need to realize this fact and modify 
expectations accordingly. Limiting the length of time for financial 
support of graduate students may also help speed up completion rates. 
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HANDLING MAJOR CHANGES (9th, pp. 8 - 10) 


The modern university places increasing emphasis on scholarship and 
research in all areas of endeavor. For some disciplines this is a relatively 
recent development, and they are faced with problems of transition from 
earlier emphases mainly on teaching and service to meeting the new 
expectations. Such major shifts involve not only modification of aims and 
programs, but also reorientation of staff. Unit Review Committees have 
pointed out that such changes are difficult and that they require long periods 
of careful management if they are to succeed. 


A quotation from one such committee report puts the problem succinctly: 


There are some obstacles to achieving the goal of becoming a leading 
research-oriented (academic unit). Many schools have learned that such a goal is hard 
to achieve and maintain. While the goal has been articulated, generally it is not clear 
that the strategies needed to achieve the goal have been well developed. We emphasize, 
however, that it is not an easy task to re-orient a major academic unit and that it takes 
time to do so. Moreover, in changing the reward system to reflect the new priorities it 
is also essential to treat fairly and utilize fully the individuals whose careers have 
developed under the previous set of expectations. . . . We note that a considerable * 
amount of administrative energy has been devoted to dealing with the problems related 
to tenured faculty who do not meet the Faculty’s new, higher standards for research 
performance. Administrators should focus on the better utilization of such individuals 
through teaching, administration and other activities that are valuable to the Faculty. 


MISSION STATEMENTS (8th, p. 7) 


Unit Review Committees have frequently recommended that departments 
develop Mission Statements. Although PACCR itself is lukewarm toward 
Mission Statements we have to agree with the thrust of Committee 
comments, namely that, given the broad boundaries of your discipline and 
given the wide range of important and desirable things to do, which are 
essential? Obviously you cannot do, or be, all things; so decide what you 
should do, and marshall your resources accordingly. In a period of serious 
financial restraint what activities must be cut back to permit continuation of 
others, or allow movement into a new area? 


NONSTANDARD UNITS (9th, pp. 11 - 12) 


Nonstandard units such as Institutes and Centres are usually attached to 
departments or faculties, but occasionally are free-standing. These units may 
find themselves unusually exposed and vulnerable in times of financial stress, 
since they have no designated spokesman to make their case in the allocation 
of resources. On the other hand, a particularly persistent director may lobby 
more successfully for funds than is warranted by the program. Nonstandard 
units may exist for good reasons, and in that case, they should neither benefit 
disproportionately nor suffer because of their special status. 
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PROFESSIONAL DEPARTMENTS (5th, p. 16) 


At the University of Alberta we have a mix of the arts, humanities, and 
sciences and a group of professional departments and faculties devoted to 
preparing individuals for vocations ranging from business to clinical 
medicine, from education to agriculture, from engineering to law, and more. 
All units comprise a mix of activities: teaching, research, and scholarly activity 
and service to the profession and the community. In a university it is 
essential that each of these areas of activity be promoted. What is not so clear 
is the proportion of each that might be appropriate for individuals in the 
various disciplines. In the professions, teaching and research activity tend to 
be modified by the demand for service. What is an appropriate amount of 
time that should be spent in providing services—with or without 
remuneration—in the overall workload of an individual, a department, a 
faculty? Some such involvement of service is undoubtedly necessary. A whole 
range of complex issues grows out of this need. 

It may be asked, therefore, whether the usual emphases on teaching, 
research, and scholarly activity in assessing merit are entirely appropriate for 
professional departments and faculties. The inclusion of professional service 
and the weight assigned to it should be given careful consideration. 


RENEWAL IN TIME OF REDUCED RESOURCES (4th, pp. 9 - 11) 


Universities are characterized by activities that conserve and advance 
knowledge activities, which exist in an overall state of dynamic equilibrium. 
Times of social and economic expansion in society are accompanied by 
growth in research and scholarship at universities. Resources allocated 
encourage pursuit of intellectual interests, and the stimulation of novelty and 
opportunity pervades. In periods when real increases in resources may fall 
even below rates of inflation, the response by universities is to stabilize and 
to consolidate former gains. Resources become scarce, staff become less 
mobile, and intellectual stimulation becomes more difficult. The possibility of 
stagnation grows. 


Constructive response to restricted access to resources is not simply —or 
even mainly—a matter of finances. Curriculum modification and updating, 
program revision, and staff reassignment are examples of changes that carry 
minimal financial costs. Creative use of staff vacancies that come through 
retirement or resignation can materially affect overall program development 
and morale. 


REPLACEMENT OF RETIRING STAFF MEMBERS (5th, p. 6) 


Unit Review Committees have often commented on the fact that a 
significant number of staff members in a given department will be retiring in 
the next few years, and increasingly they have addressed the question of how 
these members should be replaced. A frequent recommendation is that a 
department not replace a retiring staff member with a person in the same 
area of specialization. The reasons for these recommendations vary. 
Sometimes committees indicate that new fields have come to the fore in the 
last ten years or so and that a department should move into new areas, or 


Is 


PART II 


that there should be a shift of emphasis and that relatively neglected areas 
within a department should be strengthened. 


PACCR has noticed that departments generally react defensively to such 
recommendations, usually because of internal pressures. There are, after all, 
vested interests in departments. It is perhaps impossible for a department 
alone to make decisions that will mean major shifts in direction. The 
University as a whole has a stake in the long-term development of individual 
departments. 


PACCR would therefore like to recommend that Unit Review Committee 
comments concerning future appointments or directions be given to 
appointment committees, so that the observations of the Committee can be 
fully considered. 


USE OF RESOURCES (9th, pp. 15 - 17) 


Unit Review Committees have commented that, in an environment of 
decreasing overall resources, the University and its units need to examine 
their operations carefully to ensure that resources are employed most 5 
effectively. They point to three areas in particular: (1) cost recovery; (2) 
restructuring of curriculum; and (3) assigning support staff. Cost recovery 
recommendations refer mainly to payment of many research costs from trusts 
and grants, rather than from departmental funds. Curriculum restructuring 
suggests that there are overlaps and redundancies that should be removed, 
and that outdated material should be weeded out, thus reducing the number 
of courses offered and realigning teaching loads. It should be possible to 
reselect and reorganize content to reflect the current state of knowledge and 
the needs of students as they prepare for careers. The concern about 
assigning support staff is that they should be available to assist a whole 
group of researchers rather than being assigned exclusively to one or two 
persons. More efficient employment of time and abilities is the aim. 


SERVICE COURSES (8th, p. 15) 


Departments in the Faculties of Arts and Science have long ago accepted 
the principle of service to other faculties. That, however, is not always the 
case in professional faculties. Unit Review Committees have on a number of 
occasions commented on the difficult experiences by some departments 
where they need service courses. Some departments have felt compelled to 
appoint staff members to teach such courses. Review Committees have 
generally disapproved of such a practice. As an example, the Faculties of 
Rehabilitation Medicine and Nursing are ones that have often felt compelled 
to appoint members of their own faculties to teach the subjects of Physiology 
and Anatomy. 


SESSIONAL APPOINTMENTS (6th, pp. 4 - 5) 


Several review committees have commented on the often unsatisfactory 
arrangements involved in sessional appointments, particularly in large 
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departments such as Mathematics and English. PACCR recognized the 
complexity of the problem, and recommended that the University tackle the 
problem in a systematic and comprehensive manner. 


SMALL FACULTIES - SALARY AND PROMOTIONS 
COMMITTEES (FSPC) (10th) 


Small, nondepartmentalized faculties encounter special problems in the 
structure and operation of their FSPC’s. Normally, the dean sits as the chair 
of the FSPC. But in these faculties it is the dean who makes recommendations 
to the committee. It seems inappropriate, therefore, for the dean to be in the 
chair for the discussion of and decision on the recommendations. Merely 
selecting someone from the Faculty Council, who comes from outside the 
Faculty, seems to be somewhat less than satisfactory. General Promotions 
Committee should examine the issue, and recommend suitable changes to the 
Board-Staff Agreement. PACCR suggested that the dean of one of the large 
faculties— Arts, Science, Education—could be asked to nominate a department 
chair with experience in such matters to take the chair of the FSPC in the 
small faculty in question. 


TEACHING AND RESEARCH (6th, p. 3) 


Unit Review Committees have on a number of occasions addressed the 
argument that superior teaching alone should be sufficient to secure 
advancement and promotion to the rank of full professor. In every case they 
have rejected this position. 


ACADEMIC WORKLOADS 
a) Teaching Workloads 


Almost from the beginning of the reviews for academic units, 
comments were made in self studies about heavy teaching workloads, 
often in comparison with selected other departments. Early on we were 
of the opinion that, while some claims appeared valid, others were 
probably not. Our first attempt to come to grips with the problem was 
reported in the Second Annual Report, but in the Sixth an extensive 
examination (pp. 12-22) was reported involving a wide variety of 
scenarios and assumptions. In our attempt to obtain a valid relative 
indicator of teaching workload for regularly scheduled activities, we kept 
in mind the following items. 


@ Estimates of teaching loads are necessarily confined to the quantitative 
aspects (number of interactions) and cannot reasonably include the 
qualitative aspects (effectiveness of interactions). 


© Under the quantitative heading all scheduled teaching activities for a 
unit should be included: number of lecture hours and number of 
students in each course or section; similarly, number of hours of 
scheduled nonlecture activities and number of staff and number of 
teaching assistants. 
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b) 


@ A distinction needs to be made between the teaching load of a unit in 
the absolute sense and the relative load compared with other units. 


® The absolute values of load are not readily accessible. Nevertheless, 
the comparative values of one unit relative to another can be determined 
with a useful level of certainty provided sound assumptions are used to 
condense numerical information. 


@ The objective is to condense the data to produce a single number that 
represents the relative load of a unit in a given year. 


@ The indicator of the relative per-capita teaching load is the 
appropriately weighted sum of students, lectures, labs, and other 
nonlecture items divided by the number of staff plus some proportion of 
the number of teaching assistants. 


@ The relative weights assigned to numbers of contact to student hours 
(lectures and nonlectures) is the single most important factor for 
judgment. The relative weights assigned to lecture and scheduled 
nonlecture activities is probably next in significance, and finally, 
judgment must be exercised as to the weight assigned to the 
contributions of teaching assistants relative to regular staff. 


@ Since departments are highly diverse, the models adopted for data 
condensation must be as consistent as possible with reality. For diverse 
departments the process of data condensation to produce a single 
number must be validated insofar as possible by successful application 
to an appropriate set of extremely diverse departments. 


@ A level of uncertainty is necessarily associated both with a number 
representing the relative teaching load for a unit and with comparisons 
with other units. If a rigorous validation test has been passed, the level 
of uncertainty in a comparison should not be excessive. A reasonable 
estimate of reality should be attainable. 


@ Valid numerical comparisons of loads should be used as an aid to 
common sense, not a substitute for it. The information should not be 
over-interpreted or used inappropriately, for example, as the major basis 
for allocation of resources. 


It is clear that, depending on the assumptions made, it is possible to 
obtain virtually any result concerning relative ranking that one sets out 
to obtain. However, not all assumptions are equally valid. Our objective 
has been to obtain an indicator that is fair and equitable and one that is 
based on assumptions with high validity. 


More detailed comment on the matter of teaching workload and its 
validation is given in Appendix A Campus Reviews. A table for obtaining 
weighted average teaching loads for classes ranging from small to large 
and for lecture and nonlecture scheduled activities is given in Section III, 
subsection 3.7. 


Relation between Teaching and Research 


The question of workload is one focus of comment in self studies and 
Unit Review Committee reports. Workload for academic personnel 
includes teaching, direction of graduate students, research, professional 
responsibilities, and administrative activities. In particular, teaching vs 
research or teaching and research have come in for much comment. 
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No sharp dividing line can really be drawn between these two 
functions. As those instructed become more and more competent in 
their subject, the activity grades imperceptibly from formal teaching 
through informal instruction and motivation to pure research. Thus, 
while recognizing that teaching includes much more, in a narrow 
context we defined it to include only the activities associated with the 
conduct of regularly scheduled classes (mainly undergraduate); research 
then includes the nonscheduled teaching of graduate students and 
postdoctorals as well as other activities associated with research. 


It became clear that in many departments there are some commonly 
held perceptions about the two functions of research and undergraduate 
teaching. A frequent type of comment from departments in the reviews 
to date is to the effect that undergraduate teaching loads are so heavy 
that research activities are inhibited or impossible. For example, “We 
have abnormally heavy teaching loads and therefore too little time for research 
activities.” Similarly, some Unit Review Committees comment as follows: 
“Excellent teaching is a major strength of the department; it is also a 
considerable burden for research-oriented faculty and tends to blunt research 
creativity.” “The heavy teaching load . . . has inhibited research activity.” In 
this context the two activities are seen as antagonistic. 


Another perception is that high competence in research automatically 
endows one with competence in teaching. In this context the two 
activities are seen as complementary. 


It has seemed essential to attempt to determine the relation between 
research and particularly undergraduate teaching. Our comments are in 
Appendix A (Campus Reviews, Section II, subsections 3.3 and 3.4). 


Teaching and research are topics on which it is difficult to obtain 
objective data; at the same time it is a mistake to theorize before one has 
appropriate data. One observation we made was that there are 
departments claiming to have exceptionally heavy teaching loads who 
have members with teaching workloads well above that of their 
colleagues. Yet several of these same individuals are among the most 
highly productive in research. 


Reduction of teaching duties would not seem an obvious route to 
increased research activity, just as for most, less research would not be 
the road to more effective teaching. 


In Campus Reviews we present a rank correlation diagram showing the 
relation between undergraduate teaching competence and research 
productivity for a group of academic staff. The correlation coefficient 
relating the two activities was neither negative nor positive but actually 
zero. To summarize, research and teaching are not completely separable 
into neat categories; each contains components of the other. An attempt 
to separate the function of research from that of teaching does not seem 
to be so productive over the long term as a mutually reinforcing 
combination: that teaching can be a major source of research ideas and 
research a major source of teaching content. 
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INTRODUCTION 


In 1980 after about three years of discussions, including building on 
knowledge from other universities, the review concept for the University of 
Alberta was developed and agreed upon. The review process began with the 
formation of an advisory committee to the President, now called the 
President’s Advisory Committee on Campus Reviews (PACCR). The PACCR 
mandate is to conduct reviews of all campus units, academic and 
administrative/support units. This report has been prepared to provide 
information to departments, review teams, and to others interested in the 
review process at the University of Alberta. Although it is not a 
comprehensive presentation of all aspects of a review, we trust that it will be 
useful. 


Section I, “A Philosophical Approach to Reviews,” is based on a report 
given by the Chairman of PACCR to the Planning and Priorities Committee in 
September, 1981. We regard this section as important, since it is essential that 
the philosophical basis of the review process be understood if positive results 
are to be obtained. 


Section II outlines some details of the process of the reviews. 


Section III, “Academic Workloads,” is a revision and updating of a 
composite of three presentations given in 1982 by the Chairman of PACCR to 
the Deans’ Council, the Chairmen’s Council, and the Planning and Priorities 
Committee. Athough this section may give the impression of undue emphasis 
on the teaching part of workloads, it is simply an attempt to come to grips 
with an issue that is subject to manipulation. 


Section IV is derived from the Seventh Annual Report of PACCR and 
summarizes impressions and conclusions reached after the seventh year of 
conducting reviews. 
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SECTION I. A PHILOSOPHICAL APPROACH TO REVIEWS 
1.1 Objectives 


To attain the maximum positive result, there must be (1) a clear 
understanding of purpose, (2) sound procedures and instructions, (3) the 
right climate, (4) top-quality self studies and review teams, and (5) 
appropriate follow-up. 


In the broad sense the purpose of reviews must surely be to seek ways of 
serving the public interest as effectively as possible. Reviews in various 
universities have been initiated for many reasons: (1) to find programs to 
terminate, (2) to justify termination of particular programs, (3) to respond to a 
crisis situation in a particular area, (4) to fulfill accreditation requirements, (5) 
to respond to an imposition from outside, and (6) to enhance constructive 
change. The last reason is the one applicable to the University of Alberta. The 
reviews were self initiated. We wish to attain the maximum positive result 
from a systematic, non-crisis look at every unit in the University. 


If purpose is not clearly understood and agreed to, it will be difficult to 
carry out a successful program of reviews. Instructions and advice need to be 
given to departments undertaking a review as well as to the review teams. 
The nature of the instruction depends on and follows rather directly from the 
purpose. With purpose clearly understood, many operational aspects follow 
logically. 


1.2 Operational Aspects 


In an operational sense, attainment of a specified purpose embraces at 
least three components, albeit not of equal importance. 


One minor component is that of measurement of service provided. A first 
instinctive reaction is to measure, to be objective, to be scientific. 
Measurements can deal with innumerable items: total weekly student hours, 
total staff, total budget, WSH’s/staff member, $/WSH, refereed 
publications/year, entries in a citation index/staff member, and many others. 
Weighting factors could be introduced so that the overall contribution could 
be calculated for the department. Some faculties actually attempt to measure 
the worth of individual staff members for purposes of salaries and 
promotions decisions. Such reliance on measurements is somewhat akin to 
numerology. To paraphrase Leon Rosenburg, “Measurement doesn’t deal with 
the complex quality called ‘humanness’ any more than it does with such equally 
complex concepts as ‘love’, ‘hate’, or ‘trust.”’ 


If the purpose were mainly measurement, then what kind of instructions 
would be given? After considering the matter, one’s conclusions would 
probably be that a department should be instructed to collect a variety of 
data. PACCR could assign the weighting factors to the various components of 
the data. It should not then take long to complete the review exercise for the 
entire University. A technician could be hired for a year, and with the aid of 
the data, weighting factors, and a minicomputer, it could quickly be brought 
to completion. 


The members of PACCR have carefully considered the nature of the data 
that departments might report. They have concluded that as a minimum 
there should be data giving budgetary information and, for academic units, 
data on aspects relating to staff and students, including research and 
scholarly productivity. 
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PACCR has gained the impression that those who have had little 
opportunity to think about reviews are not likely to have moved far beyond 
this first step of measurement. 


A second operational component is evaluation of service provided. 
Evaluation involves the collection and examination of information and the 
exercise of judgment. Thus, in the examination phase evaluation could indeed 
include measurements, to be followed by the exercise of informed judgment. 
That is the key to evaluation — informed judgment. 


If the purpose were mainly evaluation, the department would be 
instructed to obtain a variety of carefully specified information useful for that 
purpose. For the review team a detailed list of questions would be provided. 
All potentially important aspects leading to a valid evaluation would have to 
be covered. Once embarked on the route of detailed questioning and data 
gathering, no stone should be left unturned. One would have to include 
questions for all potentially significant aspects. 


Attempts to obtain a better evaluation lead to more questions being 
devised — as many as 500 at one university. Such a blizzard of questions 
must surely snow under those who are queried. The more questions we give 
departments and review teams to answer — the more busy work — the less 
wisdom we are likely to be able to draw upon. Steering committees in many 
universities have not been fully able to overcome the barriers leading to the 
next stage. Some trade-off is necessary: possibly to lose something in the way 
of measurement and evaluation in order to get at the larger issues. 


A third component is that of program improvement — to consider how to 
do things better — in today’s world to obtain more for less. This component 
is least easily defined. When reviews were first considered here, the intention 
was that the main thrust be in the area of measurement-evaluation, with 
budget changes in mind. It was to be a sort of policing action. A change in 
philosophy took place. 


Whatever the arguments, the Folio (weekly publication of the University) 
article (Folio, February 1980) set out the philosophy for our reviews. Four 
excerpts convey the sense and help define program improvement. 


. compelling objective is preparation of a reviewing procedure that will go as 
far as possible toward guaranteeing the usefulness of those reviews assisting in 
developing a clearer picture of our selves. 


.. . we make the assumption that firm and meaningful decisions can in fact be 
made on the basis of informed judgment of academic quality . . . reviews are not 
conceived in the spirit of some sort of policing action . . . where good things can 
be encouraged and where changes will capitalize on existing strengths. 


The collecting of data would improve quantifiable knowledge . . . but we need 
good-quality information even more... a great range of benefits and uses going 
far beyond what can be derived from tables and measurements alone. 
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PACCR is an advisory body to the President. The President has made his 
position clear in Folio: 


We are not setting out to find those departments or those faculties which should 
be deleted from the University. Each individual has his pet unit and the unit he 
would be ready to push aside. 


. . our aim is not to identify a unit or two to be made a kind of 
institutional scapegoat, but rather for us to help each other to identify strengths 
and to identify weaknesses. 


A reasonable conclusion is that the mandate of our committee is that of 
program improvement — to catalyze constructive change and adaptation to 
continually changing circumstances. 


With program improvement as the main purpose, what kind of 
instructions are appropriate? How can we go far beyond what can be derived 
from tables and measurements alone? 


With program improvement preeminent, the instructions we use are more 
global than specific. To obtain the maximum positive benefit, we ask a 
department to open up both to itself and to the review team. Any review 
poses some level of threat. A mature self examination will be enhanced if the 
process and attitudes are perceived to be fair, sensitive, and respectful. The 
expectation of the possibility of constructive change is a strong antidote to 
embarrassment from revealed weaknesses. 


Each department is asked to tell its story, with a strong forward-looking 
emphasis. Upon completion of the self study, the review teams are then 
chosen in consultation with the departments. From the review team we also 
look for a strong message and maximum impact. This objective is more likely 
to be obtained if members of the team are given freedom to investigate and 
report on what they find. 


In summary, the three operational components vary in importance. 
Program improvement must include some component of evaluation, which 
usually includes the need for some critical measurements. Measure and 
evaluation, yes, but much more. 


ZY, 


VHECKREVIEW PROCESS 


SECTION I. THE REVIEW PROCESS 


The entire process, as it has been developed at the University of Alberta, 
is summarized in Figure 1. The procedure begins with the department's 
engaging in a self study and preparing a Self Study Report. This is followed 
by an on-site visit by a Unit Review Committee (URC). The committee’s 
report, which is usually received a few weeks after the visit, is sent to the 
President for clearance and then to the department for their response. Once 
all three reports (the Self Study, the URC Report, and the Department 
Response) have been received, the members of PACCR consolidate and 
summarize their findings. Finally, they meet with the head of the unit and 
later with the President. The President discusses outcomes of the review with 
the Chairman/Director, Dean, and appropriate Vice-President. This group 
then meets with the Planning and Priorities Committee. The Vice-President 
(Academic) or Vice-President (Administration) is involved in long-term 
follow-up a year or two later. 


2.1 Self Study Phase 


To begin a review, the members of PACCR meet with the members of the 
department to present a brief background on the purpose of a review, outline 
the procedure, offer advice, and answer questions. 


During the self study such aspects as history, current status, pending 
changes, future prospects, strengths, and limitations of the department 
should be critically examined. How the discussions on these topics are 
organized is for the department to decide. As many as possible should take 
an active role in examining pending changes and future prospects. 


In general, the focus should be on key issues, a frank and balanced 
consideration of both strengths and areas for improvement, and strategies 
proposed for future changes and commitments for such changes; most 
important is that the Self Study Report convey a sense of the larger issues. To 
accomplish these changes, the roles that individuals or groups will assume 
should be agreed to. The experience in other universities and here is that the 
most successful studies are those that involve a majority of members of a 
department. 


Out of the self study is produced a report that serves as the primary 
document for the Unit Review Committee. The report should be well _ 
organized, clearly written, and complete yet concise. It should not reflect a 
total preoccupation with its preparation but should mainly arise out of the 
self study. The quality of the report will probably be enhanced if a small 
steering group is responsible for its preparation and drafts are circulated to 
the members for comment. Possibly a draft of the history of the department 
can be prepared by one or two members, and a draft statement of the current 
status assembled by several. 


The report should be looked on as being data-assisted, not data-driven; 
certainly, data supporting points of view should be presented. A certain 
amount of statistical information should be included concerning operating 
budget, staff, students, and funding. 


Although PACCR desires that maximum freedom be given in the 
development of the Self Study Report, there are a few constraints. For 
example, the report should contain the staff profile in an appendix or, if 
desired, for academic departments curricula vitae can be separated from the 
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Figure 1. The Review Process in Detail 
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main report to ensure more restricted circulation. For CVs the members of the 
department should adopt a uniform and efficient format that summarizes the 
important information. 


Since the report carries most of the message and impact, its quality not 
only determines the extent to which it is useful to the Unit Review 
Committee, it is also probable that it influences the efficiency of the 
review-team efforts; when the report is clear and thorough, the team can get 
to important matters expeditiously. 


Self studies and the preparation of Self Study Reports involve substantial 
effort and are bound to be time-consuming, particularly for the 
chairman/director. The reports, by and large, have been well done. They tend 
to be realistic, and concerns, strengths, and problems are usually clearly 
defined. Balance must be maintained between the effort involved in carrying 
out a review and the benefit obtained from it. The impression of the members 
of PACCR is that the better the quality of the self study, the better the work 
of the review team and the more helpful its report is likely to be. 


2.2 The Unit Review Committee 


The Unit Review Committee normally consists of four members, two from 
outside the University from the same discipline, one internal member from a 
related discipline, and one internal member from the University-at-large. For 
some units a fifth member “from the profession” may be included. The 
members of the Unit Review Committee are approved by the President acting 
on advice from PACCR. The composition of the review team is vital to a 
competent review. Also important is that all members of the team have 
credibility both inside and outside the unit reviewed. In Folio this statement 
appeared: 


The review team must have credibility among those being reviewed and those who 
will consider the recommendations. 


The department is requested to provide four to six nominees for the 
external members of the team and also nominees for the internal-related 
member. Department members should begin to consider possible nominees at 
an early date. PACCR suggests nominees for the University-at-large position 
and may consult with others to obtain comments on the appropriateness of 
nominations. Nominees should have no direct connection with the 
department concerned (spouse of a staff member or an instructor in the 
department, for example). 


For nominees it is important that each has achieved a high level of 
excellence in his/her chosen field and is characterized by objectivity and 
trusted judgment. In general, knowledge about these factors will be no 
problem for the internal-related and the University-at-large members. For the 
external members, PACCR can make some discreet inquiries, but most 
important, the chairman meets with PACCR and should be prepared to 
provide evidence for these characteristics. Evidence of achievement of a high 
level of scholarship for a scientist, for example, can often be appropriately 
obtained from the entry in the American Men and Women of Science. The 
trusted-judgment aspect may be more difficult. Evidence for this might 
include, for example, successful discharge of duties in an administrative 
position, editor of a journal, member of a granting agency, or advisor to 
government. 


en 


APPENDIX A 


Part of the rationale for the selection of the several review committee 
members is that each should bring some special viewpoint to the 
deliberations. In that regard the role of the two external members probably 
needs no explanation. 


The internal-related member of the team should know a great deal about 
the department and thus will inevitably be well known to many of the 
departmental members. This member brings special ability to interpret for the 
other members of the team the particular setting and development of the 
department in this University. 


The University-at-large member acts in certain respects as representative 
of the wider University community and should pose questions that reflect a 
broader point of view. Responses to a review are expected to come in part 
from individuals that normally do not have expert knowledge in the field 
being reviewed. The URC Report should therefore be written partly with the 
wider perspective in mind. The University-at-large member is deliberately 
chosen to be one who is at some distance from the field and therefore 
unlikely to be well known to members of the department. 


2.3 The On-Site Review 


The on-site review normally takes place over a two-, three-, or four-day 
period, depending on the size and complexity of a unit. Each reviewer is 
supplied ahead of time with several items including the Self Study Report 
and this background report about reviews. During the site visit arrangements 
are made for direct interchange of information with the head of the 
department, members of the department, students if appropriate, chairmen or 
directors of related departments, Dean, and President. About one third of the 
time is spent in the department, one third outside the department and one 
third in committee discussing findings and reaching conclusions. 


For a typical three-day on-site visit the main block of time for the 
department begins at noon on the first day and includes the afternoon plus 
the afternoon of the second day. Debriefing time is scheduled with the 
department chairman during the third afternoon. The detailed schedule for 
the first and second afternoon is arranged by the department. For academic 
units the arrangements must include interviews with undergraduate and 
graduate students. The schedule should also include an opportunity for 
individual staff or groups of staff to meet with the committee, so that they 
receive direct input from as wide a range of departmental personnel as 
possible. 


Other interviews are arranged by PACCR. Here PACCR wants the advice 
particularly of the department, but also from others as to who could most 
appropriately provide informed input. The members of the Committee of 
course have the freedom to seek information from any of a variety of sources. 


The members of the Unit Review Committee are not given detailed 
directions and questionnaires, but rather, broad general instructions. The 
instructions are as follows (references to teaching and research being deleted 
for administrative/support units): 


Without intending to restrict the scope of the review, we should like to have 
drawn to our attention what, in the opinion of the members of the Unit Review 
Committee, are the strengths and weaknesses of the department's teaching and 
research programs, an assessment of the staff and staff workloads, the adequacy 
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of resources provided, the effectiveness of the organization of the department, and 
the relations of the department with others. In particular, we should appreciate 
having drawn to our attention any opportunities the Unit Review Committee 
perceives for improving the department’s programs, its relations with both 
internal and external organizations, and for better utilization of resources 
available. We also ask for your judgment of the department’s proposed plans for 
the future. 


As can be seen from this instruction, there are no strict boundaries to the 
matters the team may consider. PACCR also provides academic departments 
with an advice sheet to be given to students supplying input to a review 
team. 


To the University and wider community an invitation to provide input to 
the review team is extended by way of a notice in Folio, such as in the 
following example. 


Campus Reviews 


As part of the process of carrying out systematic reviews under the 
guidance of the President’s Advisory Committee on Campus Reviews 
(PACCR), the Department of Occupational Therapy is completing the 
self-study phase of the review process. The other major phase of the 
process is the site visit and assessment by the Unit Review Committee. 
In preparation for this second phase, PACCR invites written comment 
prior to 24 October 1986. Submissions are for the use of the Unit Review 
Committee and will be held in confidence by the members of the 
Committee. 


Submissions should be sent to: Mrs. Shirley Moore, Coordinator, 
PACCR Office, W4-13 Chemistry Building. Attention: The Department 
of Occupational Therapy Review Committee. 


2.4 The Time Question 


When review teams are on site, departments normally want as much of 
their limited time as can be allowed. The problem of limited time is a subject 
of continuing comment, particularly by departments. 


The time scheduled for on-site activity depends on several considerations. 
Size of the unit under review is obviously one. Similarly, departments with a 
set of complex responsibilities require more time. There are probably also 
faculty differences. 


Possibly not so obvious in the matter of on-site time are the conflicting 
interests of the department and the reviewers. Those nominated are highly 
respected, mature individuals who usually have many demands on their 
time. Only by setting review dates two or three months in advance is it 
possible to arrange a time acceptable to all. 


Our experience is that review teams could always use more time. To 
allow the members of a team to concentrate to the maximum extent on the 
review, invitations of a social nature to the external members are discouraged. 
It is of course important that the most efficient use be made of the time 
during which the review team is on site. 
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2.5 The Response Phase 


When the URC Report is received and given clearance by the President, it 
is sent to the Chairman/Director and the appropriate Dean or Associate 
Vice-President. The Chairman/Director is asked to circulate it to members of 
the unit for close study. The unit is asked to prepare a response in which 
they can agree, differ from, or add to the findings, and to undertake 
implementation of recommendations as appropriate. 


In most URC Reports some recommendations should be brought to the 
attention of the President, some to the faculty, and some to the department; 
some are appropriate for joint consideration. The scope of a department's 
response is not restricted. However, it is suggested to the department that it 
should address those matters on which it can take action. After receiving the 
Response and summarizing the reports, PACCR meets with the head of the 
unit. PACCR is not part of the decision-making structure of the University; 
therefore during this meeting the discussion takes the form of observations, 
impressions, and advice. Following this meeting with the head of the unit, 
PACCR submits its summary together with the three reports to the President. 


2.6 Action by the President 


After the President has received all the information and met with 
PACCR, he then meets with the head of the unit, the Dean or Director for 
the unit, and the appropriate VicePresident. In the case of academic 
departments the Dean of Graduate Studies is included. This group considers 
actions that are appropriate for joint consideration and later meets with the 
members of the Planning and Priorities Committee. 


Inevitably, there is a tendency to focus on present and medium-term 
matters, and such a focus is important. The disposition phase should, 
however, include some procedure for enhancing the effectiveness for 
longer-term matters that can otherwise be neglected. At the beginning it was 
agreed (Folio, February 1980) that a Vice-President should have responsibility 
for longer-term monitoring. About a year and a half after the review has been 
processed by the President and the Planning and Priorities Committee, the 
Vice-President reconsiders some of the main items, short-term and long-term, 
whose disposition may need to be reviewed at this later time. 


In conclusion the chart in Figure 1 can be simplified to show the major 
steps in a review — Self Study Report, On-Site Review, Unit Response, and 
Administrative actions and follow-up. (Figure 1A) 
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Figure 1A. 
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SECTION III. ACADEMIC WORKLOADS 


3.1 Definition of the Issue 


The question of workload has had and continues to have a high profile in 
many of the departments reviewed. The perception of what constitutes the 
academic workload varies. For some, particularly those external to the 
University, workload and teaching workload are synonymous. Workload, 
however, includes a variety of responsibilities: 


(1) the teaching of regularly scheduled classes, classified as lectures, 
laboratories, seminars, and “other” 


(2) the direction and supervision of graduate students 
(3) research and original scholarly work 


(4) service and professional responsibilities such as to the community at 
large and to scholarly societies 


(5) administrative responsibilities 


Conscientious academic-staff members during their working hours, when 
not occupied with, say, teaching a regularly scheduled class, are engaged in 
one of the other categories of activity. An attempt to arrive at a numerical 
indicator of workload would be complex and probably unproductive. In 
Utopia University the members of each department would be fully and 
effectively occupied with a workload of teaching, scholarly, and service 
activities, and there would be no need or inclination to make comparisons 
with colleagues in other departments or to score points by underrating the 
contributions of others. However, our experience is that interdepartmental 
and interfaculty comparisons are often made with a set of ground rules 
selected by the unit making the comparison. Comparisons are often made on 
the teaching part of total workload, with the unstated implication that 
workload and teaching workload are synonymous. 


The teaching part of workload itself embraces both regularly scheduled 
and nonscheduled activities. To attempt to evaluate the nonscheduled part of 
teaching would seem to be impossible. A few years ago, in view of the 
number of interdepartmental comparisons that were being made, PACCR 
attempted to examine some of the main features of the teaching-load 
problem. Briefly, our conclusion was that a combination of regularly 
scheduled contact and student hours was a reasonable indicator of teaching 
load. Further, we warned that the whole question of teaching loads must be 
approached with care, and conclusions should be drawn with caution. 


3.2 Workload and Resources 


As a result of fiscal restraints on the University as a whole, workloads in 
teaching and research may be affected. Support has decreased, particularly 
for the supplies aspect of the budget. Departments understandably strive to 
maintain or increase their share of the economic pie in appeals for support. 
The following are some quotations from Self Study Reports: 


Relative to every other comparable unit on this campus, this department is 
grossly understaffed and underfunded. Additional time for improving our 
research performance would be one important result from a fair allocation of the 
University resources. 
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We are overworked to the point of stress. There is a feeling that the University is 
forcing too hard a pace. 


Our future prospects are alarming unless we are allowed further manpower. 


There can hardly be a stronger case than ours anywhere on this campus for extra 
staff plus major infusions into supporting budgets. 


Entirely understandably, teaching departments want to forestall increases 
in workloads. In many Self Study Reports the case is made for infusion of 
additional resources. The claims are usually supported on the basis of heavy 
teaching workloads. The claim for additional staff is usually justified by 
numerical comparisons with other units within or outside their faculty. In the 
reports to date, four kinds of heavy workloads have been described as bases 
for the need for additional resources: high weekly student hours (WSH) per 
full-time equivalent (FTE); high contact hours (WCH) per FTE; heavy 
responsibility for laboratory activities; and occasionally a heavy research load. 
In the comparisons the ground rules inevitably favor the department choosing 
to make the comparison. 


Obviously, the main function of PACCR must remain that of carrying the 
reviews through the self study, review-team, and response stages. The 
President, however, has requested that we forward to him not only the three ~- 
main documents arising from the review, but also our observations and 
comments. What should be our advice to him on the whole subject of relative 
workloads? At a meeting (1982) with the President’s Advisory Committee of 
Chairmen the following opinions were expressed: 


Teaching loads need not be equalized across the University. Many factors are 
operating. 


Undergraduate teaching loads should not be the principal basis for reallocation of 
resources. 


Other activities such as research should be included. 


If research is included, it should be demonstrated productivity in contrast to 
expressed desire. 


3.3 The Research-Teaching Relation 


No sharp dividing line can, or should, be drawn between the two prime 
and inseparable functions of teaching and research. In the broad context, in 
considerations of workload the separation of teaching and research may 
introduce an inappropriate artificiality. As those instructed become more and 
more competent in their subject, the activity shades imperceptibly from 
formal teaching through informal instruction and motivation to pure research. 
Thus, although teaching includes much more, in a narrow context it can be 
considered to include only the activities associated with regularly scheduled 
classes (mainly undergraduate). Research then includes the nonscheduled 
teaching and direction of graduate students and postdoctorals as well as the 
other activities usually associated with research. 


There are some commonly held perceptions about the two functions of 
research and undergraduate teaching. One type of comment from 
departments is to the effect that undergraduate teaching loads are so heavy 
that research activities are inhibited or made impossible. For example, 


We have abnormally heavy teaching loads and therefore too little time for 
research activities. 
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Similarly, review teams comment as follows: 


Excellent teaching is a major strength of the department; it is also a considerable 
burden for teaching-oriented faculty and tends to blunt research creativity. . . . 
The heavy teaching load . . . has inhibited research activity. 


In this context the two activities are seen as antagonistic. 


Another perception is that high competence in research automatically 
endows one with competence in teaching. In this context the two activities 
are seen as complementary. 


To help us make informed comment, it has seemed essential to attempt to 
determine the relation between research productivity and, in particular, 
undergraduate teaching effectiveness. There can be little question that when 
undergraduate teaching duties become excessively large the energy and time 
required to conduct research is no longer available. Similarly, commitment to 
research may preclude much effort being devoted to the teaching function. 
The question then is, “When do the loads in one area become so heavy as to 
interfere seriously with the activity in the other?” 


The topics of teaching and research are ones on which it is difficult to 
obtain objective information. It is a mistake to theorize in the absence of 
appropriate data. Some observations are pertinent. One is that we find that 
some departments, claiming to be heavily loaded with teaching, have 
members with teaching workloads well above those for a typical staff member 
of the university-at-large. Yet several of these same individuals are among the 
most highly productive in research. Another observation: most staff have a 
substantial period in the summer during which no undergraduate teaching 
duties are required. Still another observation is summarized by the data in 
Figure 2. It is an example of the variation in research productivity for a 
group, few if any of whom had a valid claim to an excessive teaching load. 
For those at the less productive end of the chart, reduction of teaching duties 
would not seem an obvious route to increased research activity, just as for 
most staff members less research would not be the road to more effective 
teaching. 


NUMBER OF INDIVIDUALS 
a 


0 a 
0-5 6-10 11-15 16-20 21-25 26-30 31-35 36-40 40 + 


PUBLICATION COUNT, 5 YEARS 


Figure 2. Number of refereed publications over a five-year period for a 
group of academic staff. 
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One of us (WEH) had intimate enough knowledge of two major 
departments to be able to obtain not only the open information but also 
confidential information and opinion concerning staff. On the basis of the 
information, each of 65 midrange and senior staff members were ranked on a 
five-point scale as to research productivity and independently as to 
undergraduate teaching competence.” 


On the basis of a composite of several indicators, individuals can be 
assigned to one of five equal-size groups. In Group 1 were those with the 
lowest productivity or competence, and in Group 5 those with the highest. 


The diagram obtained for the 65 individuals is shown in Figure 3, each 
dot representing one individual. This diagram should be examined carefully. 
Clearly, some areas of the diagram are more heavily populated than others. It 
is highly significant that, of the 25 possible combinations of rankings, every 
one is occupied by at least one individual. No simple linear relation between 
the two functions is evident. 


Interestingly, for this sample of 65 staff the overall correlation coefficient 
relating undergraduate teaching competence to research excellence is 0.0. 
That is, the correlation coefficient is neither positive nor negative. Thus in 
some situations research and teaching can be regarded as complementary, : 
and in others as antagonistic. Both effects are in operation, a situation that 
leads to a zero or low correlation coefficient between teaching and research. 


Teaching 


Research 


Figure 3. Diagram showing relation between undergraduate teaching 
competence and research productivity for a group of 65 midrange and senior 
academic staff. 


* To obtain relative rankings, several indicators were combined in each case. For example, in research 
such indicators of productivity can include number of refereed publications, size of research grants, and 
colleague judgments. 
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It would appear from Figure 3 that the most effective teachers are active 
in research. At the same time the combination of great research and great 
teaching is rare; great researchers do not often demonstrate highly competent 
undergraduate teaching. Examining another part of the diagram, it appears 
that a substantial number are marginally effective in both capacities. Among 
these are perhaps those who assert that teaching or research rather than 
teaching and research is appropriate for university staff. 


Nevertheless, we must return to the concept that research and teaching 
are not completely separable into neat categories. An attempt to separate the 
function of research from that of teaching does not seem to be so productive 
over the long term as the idea of a mutually reinforcing combination: that 
teaching can be a major source of research ideas and research a major source 
of teaching content. The most productive research may well occur under 
circumstances where the researcher is interrupted about one third of the time 
to carry out teaching duties, and the most competent teaching is carried out 
by those who are active in research. One must agree with the statement of 
one review team, 


We do not accept the notion in the Self Study Report that the University should 
recognize two groups of equal distinction: those who do research and those who 
teach. 


Research and teaching can hardly be viewed as belonging in separate 
compartments. Otherwise, in a university some individuals could be assigned 
the task of production of knowledge and others the task of transfer of 
knowledge. The two functions do, and should, interact, and each contains 
components of the other. Notwithstanding these aspects, in the remaining 
sections the teaching-load question is examined. 


3.4 Factors of Importance in an Indicator of Teaching Load 


At the beginning we were caught off guard on the question of teaching 
load. We soon concluded that, whether or not we wanted to, we had to 
address this contentious subject, in which there are vested interests. Each 
department has unique features. Depending on the assumptions made, 
almost any department can make a claim for heavy teaching load compared 
with other selected departments — almost any answer can be obtained. our 
objective was to obtain as simple an indicator as possible, while recognizing 
that departments across the University are diverse and simplistic answers | 
cannot to be expected. For valid, broadly applicable comparisons our 
assumptions are as follows: 


1) An indicator must be applicable with reasonable fairness to all 
departments across the University. We recognize that possibly for a 
single faculty or a single department simple assumptions may be valid. 
If simpler assumptions are used, then comparisons need to be 
restricted to departments whose teaching workloads have similar 
characteristics. For example, the teaching format (such as class size 
and the proportion of lectures to nonlecture activities) for Mathematics 
is much more like that for English than it is like that for Chemistry. A 
comparison of loads in English and Mathematics with simple 
assumptions would probably be valid. With the same simple 
assumptions a comparison of the teaching loads of Mathematics or 
English with those of Microbiology or Chemistry would not be valid. 
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2) An indicator must fairly take into account diversity. Thus the same 
ground rules must be applicable with fairness to such diverse 
departments as, for example, Economics, East Asian Languages, 
Microbiology, Restorative Dentistry, Civil Engineering, and Pathology. 
Some examples of diversity are mentioned here: (a) Regularly 
scheduled classes for some departments are 100% in the form of 
lectures, and for others less than 5%, with the remainder labs, 
seminars, and other regularly scheduled nonlecture activities. (b) Some 
classes are tiny and others have huge enrollments. (c) Some 
departments have no teaching assistants and others have many. 


3) A combination of contact hours and student hours leads to an indicator 
of much higher validity than either alone. The particular combination 
is a matter of judgment. Contact hours as an indicator of load is 
defective because no allowance is made for variability of class sizes. 
Student hours alone as a simple indicator is defective because it, in 
contrast to contact hours, makes too much allowance for enrollment. 


4) The relative weight to be assigned to regularly scheduled nonlecture 
activities compared with scheduled lectures is again a matter of 
judgment. The contributions of teaching assistants must be included, 
the weight to be assigned again a matter of judgment. - 


5) It is impossible to include in the indicator the commitment of an 
instructor or the effectiveness of the instruction. These of course, vary 
enormously. 


6) It is assumed that through negotiations over the decades, chairmen 
and deans have exerted continuing pressures that have tended to 
equalize teaching loads across the University. Real differences in loads 
arise or persist because of factors such as (a) the nature of the 
discipline, (b) the inability of the University to respond quickly to 
shifting teaching loads with time, (c) differences in the aggressiveness 
with which chairmen have pursued the available teaching resources, 
(d) the impact of somewhat external factors, such as the Alberta 
Heritage Foundation for Medical Research, and (e) variability in 
nonlecture components. 


7) Finally, not considered is the question of absolute loads - whether 
teaching loads throughout the University are heavy or light. 


To use either contact hours or student hours alone as an indicator of 
teaching load is simple, but in our opinion inadequate and invalid. As 
pointed out, contact hours makes no allowance for class enrollment and 
student hours makes too much allowance. A realistic simple indicator would 
lie between the two extremes — a combination of the two, which for the sake 
of simplicity, we have chosen to call a weighted average. 


Although the so-called weighted average is elementary, it is less simple 
than either contact hours or student hours alone: to obtain weighted 
averages, one needs the detailed information on enrollments for individual 
classes. 


There are many possible weighted averages. After extensive consideration 
and consultation the weighted average that appears to be realistic and valid 
takes account of contact hours and student hours in the ratio of two to one. 
Table 1 shows a comparison of the two-to-one weighted-average indicator, 
with student hours and contact hours for lecture classes ranging from tiny to 
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large. Without getting embroiled in the mechanics, the unit for weighted 
average was chosen in such a way that the numbers are expressed and can be 
interpreted as modified contact hours. As can be seen from this table, the 
weighted average gives some, but not much, credit for the tiniest class and 
more, but not excessive, credit for the largest. 


Table 1. Comparison of WSH and WCH with the 2:1 weighted-average indictor of 
load for lecture courses of 3 h/wk. 


Class Enrollment WSH WCH Wtd.Av. 


’ 6 3 i 
8 24 3 2 
Dig 81 > S 
100 300 ) 4.6 
400 1200 6) 7.4 


We concluded that obtaining a reasonable and fair estimate of relative 
teaching loads among departments is not an excessively complicated mystery. 
However, as pointed out a fair indicator of scheduled teaching load must 
recognize that some departments have almost all scheduled teaching activities 
in the form of lectures, some with large and others with small classes; others 
have a major proportion of nonlecture responsibilities, some with and others 
without the assistance of teaching assistants (TAs); and others have 
essentially no lectures. Thus an estimate of load must include the major 
numerical variables — numbers of lectures, students, scheduled nonlecture 
obligations, and staff and TAs. For sound comparisons none of these factors 
should be ignored. 


3.5 A Reexamination 


The subject of teaching load continues to arise in the reviews of some 
departments, and there has now been an opportunity to obtain responses 
concerning the approach described in the preceding section. These responses 
are to the effect that some would request that an indicator of teaching load 
assign more credit to large classes, others more credit to small classes, and 
still others that it should be more sensitive to class size. In other areas some 
understandably want more credit for scheduled nonlecture activities, or for 
scheduled lectures, and so on. Although there are similarities, each 
department differs in the nature of its teaching duties. It would probably 
simplify matters if that fact were acknowledged and teaching-workload 
comparisons were avoided. However, this option is not open to us. 


The question of teaching load has been reexamined a couple of times in the 
last few years*. The reexamination has included all departments except those in 
clinical medicine, and for a period of several years. The conclusion from the 
reexamination is a reaffirmation that a valid indicator is a 2:1 combination of 
WCH and WSH and that a fair evaluation of teaching load should include not 
only regularly scheduled lectures, but also regularly scheduled laboratory 
activities and regularly scheduled nonlecture, nonlaboratory activities. The 
indicator must also recognize the teaching contributions of graduate teaching 
assistants (GTAs). In the reexamination more accurate information was available 
for nonlecture, nonlaboratory components of the teaching responsibilities of 
departments than for the earlier calculations. 


*PACCR’s Sixth Annual Report has a major section on teaching workloads; the reader is referred 
to it for details. 
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In PACCR’s Sixth Annual Report a number of different scenarios were 
examined from among the innumerable possible combinations of 
assumptions. In one scenario contact hours were overemphasized (3:1 
weighted average); in another student hours overemphasized (1:1 weighted 
average). The TA contributions could be counted more heavily (equal to 
regular staff) or not at all. Finally in another scenario only lectures are 
counted. 


A sample of ten departments was selected whose instructional duties 
illustrate well the diversity among departments. On the basis of the preferred 
method of estimating teaching load, their loads were ranked from 1 (heaviest) 
to 10 (lightest). According to the other scenarios the rankings that would 
result were also determined. The results are shown in Table 2. 


Table 2. Teaching Load Rankings for Six Scenarios Within a Sample of 10 
Diverse Departments 


Rank Contact Student TAs TAs Lectures 
Preferred hours hours contribut. contribut. only 
scenario overrated overrated upgraded downgraded included 


Pail 31 fa papi 21 pas | 
1 (heaviest) 2 3 5 2 4 
2 3 4 2 a 6 
3) 7 Z 1 6 1 
4 1 8 3 y 10 
5 4 5 4 3 9 
6 9 | 8 5 2 
Th 6 7 10 1 8 
8 5 10 6 9 3 
9 8 8 in 10 7 
10 (lightest) 10 6 ) 8 5 


The table clearly shows that none of the five scenarios selected for study 
agrees with the preferred ranking or with each other. With but slight 
exaggeration it is possible to demonstrate virtually any ranking one chooses 
by making opportune but invalid assumptions. 


According to our best judgment, we conclude that a sound indicator of 
teaching load should continue to involve the 2:1 weighted average proposed 
earlier (the preferred scenario of Table 2) and that it should include a 
weighted-average allowance not only for lectures, but also (with enrollment 
divided by ten) for labs, seminars, and other activities. We judge that the 
contribution of GTAs should be included at half the weight of regular faculty 
members. 


Weighted-average teaching loads per staff member were calculated for all 
departments (except those in clinical medicine where data was not available). 
Data on detailed enrollments were obtained from the Office of Institute 
Research and Planning. Figure 4 shows the distribution of teaching loads for 
the University as a whole for the years 1982-85. These data are a refinement 
of the main features that were clear earlier. 
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Number 16 
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Departments 
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0) 
0 4 8 12 


Weighted Average 


Figure 4. The distribution of teaching loads for the three years 1982-85 for 
the University of Alberta using a 2:1 WCH:WSH weighted average as the 
indicator (preferred scenario of Table 2). The data from IRP for the number 
and class sizes for courses number 100-299, 300-499, and 500 and up were 
used to obtain the weighted averages. 


3.6 Empirical validation 


Any of several simple modes of comparison are likely to be valid for 
departments that are similar in class size, extent of scheduled nonlecture 
activities, and teaching-assistant contributions. However, if comparisons are 
attempted where the activities differ markedly, then simple comparisons are 
likely to be invalid. Any indicator that would give valid comparisons of 
teaching loads among departments that differ markedly in the nature of their 
teaching duties must be applicable with fairness to all diverse departments. 
The following five departments represent well the diversity encountered in 
the University: East Asian Languages and Literature, Genetics, Occupational 
Therapy, Psychology, and Stomatology. These five are highly variable in the 
nature of their instructional duties, but they also had per-capita 
weighted-average teaching loads that did not differ significantly from each’ 
other in 1985-86. Independent evidence involving discussions with the five 
chairmen supported this conclusion. 


When weighted average loads for the University as a whole are examined 
the diagram showing the distribution (Figure 4) is substantially narrower than 
similar diagrams with either contact or student hours alone. With either 
contact or student hours the distributions contain the broadening which arises 
from the real differences in loads that are apparent with the weighted 
averages, but in addition there is extra broadening coming from the 
defectiveness of either contact or student hours as indicators of teaching load. 
The conclusion is that the narrower distribution with the weighted average 
results because it is a more realistic indicator of load. 
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3.7 Weighted Averages 


Those interested in the mechanics of weighted averages should recognize that in 
the most general sense an indicator of teaching load can be set equal to 
(enrollments) *, where the enrollments include those in lectures, labs, and other 
activities. If x is zero (no dependence on enrollment), an indicator of load is simply 
that of number of courses taught, and if x is one (total dependence on enrollment), 
the number of students taught. With x having a value between zero and one, the 
indicator is a weighted average of number of courses and number of students. For 
example, with x equal to one third, the weighted average includes contact hours and 
student hours in the ratio of two to one. The relation of several weighted averages to 
each other are depicted schematically in Figure 5 as the dependence of the indicator 
on enrollment varies from zero to totality and x from 0 to 1.0. A mathematlcally 
inclined colleague has pointed out that more generally a teaching load indicator can be 
defined as the sum (over all sections taught) of terms 

Kew*(WCH)*(WSH)'* 

where K is some constant, w is a weight that depends on section type, and WCH and 
WSH are the weekly contact and weekly student hours calculated for each section. 


Weighted Averages 


No. No. 2 
Courses 22] Students 
rs i 1:3 
0 0.25 0.5 0.75 1.0 
Zero ----------------------- partial ----------------------- total 


Dependence on Enrollment 


Figure 5. Diagram to show the relations among the various indicators of teaching load. 


Table 3. Table of Weighted Averages* (2 WCH to 1 WCH) 


Enrollment Wt Av Enrollment Wt Av Enrollment Wt Av 
0.5 0.8 14 2.4 100 4.6 
i 1.0 16 pa5 110 4.8 
Le ileal 18 2.6 120 4.9 
ey ile 20 De, 130 byl 
2 iL 25 2.9 140 52 
3 1:4 30 Ball 150 5S 
4 1.6 35 3.3 160 5.4 
5 Ihe 40 3.4 170 55 
6 1.8 45 3.6 180 5.6 
q 1.9 50 SeH/ 190 Bui/ 
8 2.0 60 3.9 200 5.8 
9 pri 70 4.1 250 6.3 

10 Dep 80 4.3 300 6.7 
12 DES} 90 4.5 400 7.4 


*For courses meeting 3 h/wk. Indicator of teaching load = (enrollments) where x = 1/3. Regularly scheduled 
nonlecture activities included as enrollment/10. Weighted average expressed as modified WCH. For a class of 3 
h/wk with 27 students, weighted average is the same as WCH. 
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Table 3 lists values of the 2:1 weighted averages for a whole range of class 
enrollments to an upper limit of 400 students. A detailed calculation of 
teaching load for the hypothetical Department of Publications Engineering, 
1995-96, is given in Table 4. The department is assumed to have 10 FTE 
faculty and 8 GTAs. 


To make calculations in Table 4, for labs, seminars, and other, divide number 
of students by 10, and weight the FTE for GTAs at half the value of regular 
teaching faculty. 


Table 4. Sample Calculation of Teaching Load 


# Students 
Course # # Sections Per Section h/wk Wt.Av. (Table 3) 
Pub 200 4 120 3 4x 4.9 = 19.6 
Pub 310 55 8 DX Bitsy = ILS 
Pub 320 il 42 g) xe tsS Set 
Pub 350 1 35 3 IXGS OF =a o 
Pub 450 1 24 3 12,9 = 29 
Pub 460 i 20 3 LR = Diy 
Pub 500 1 10 3 X22 Ss De 
Pub 502 1 8 3 [xe 2 = 220 
Pub 504 1 6 3 xis = 1.8 
Pub 506 1 (2 3 G27 oe= 2.3 
Pub 508 1 14 3 (1 term only) 2AI2 = 1.2 
Pub 510 1 3 3 xen — Sele 
Pub 515 1 5 3 VSeikey = iy 
Pub 600 1 20 il 2.7/3 = 0.9 
Pub 610 1 15 2 DOS = 17/ 
labs (200) 20 24 3 MDSEULS = ZG0 
labs (310) 6 We 2 6xel 2x 2/3) = 458 
seminars (320) 4 10 1 4x10x13= 1.3 
other regularly 
scheduled 2 15 2 DANA AS Sie 
Total 88.4 


Average teaching load 88.4 (10 + 8/2) = 6.3 


3.8 A Suggestion 


For those who wish to obtain for comparative purposes an indicator of 
the teaching load for their own department, it is suggested that the reader 
use Table 3 (Subsection-3.7 Weighted Averages). For purposes of comparison 
the result obtained can be positioned on the distribution in Figure 4. To assist 
the reader there is an example of a detailed calculation of teaching load for a 
hypothetical department. The basic information needed is the enrollment in 
each class, the hours per week, one term or all year, lectures only or lab and 
nonlecture, number of staff, and number of teaching assistants. 


It is important to recognize the limitations of the estimate of a teaching 
load that: it is necessarily restricted to the quantitative aspects and cannot 
include the qualitative aspects; that the absolute values of load are not readily 
accessible but estimates are restricted to relative values; and that a level of 
uncertainty is necessarily associated with the number representing the relative 
teaching load. The level of uncertainty should not exceed 10 to 20% 
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SECTION IV. THE PACCR PROCESS IN PERSPECTIVE 


What are the benefits of the review process? That question is often asked, 
and the answers are both simple and complex. Since the involvement of 
PACCR ends with our submission of the various documents to the President, 
we are not in a position to know in detail what effects reviews have had on 
decisions and actions. Thus others are in a better position to answer the 
question, “Do reviews make a difference?” Nevertheless, it is clear to the 
members of PACCR that many important changes have been brought about 
as a result of the review process, particularly when a unit wants the effort to 
be productive. We attempt here to give examples of these changes. At the 
same time, it is not possible to produce a simple balance sheet with the 
proverbial bottom line. 


PACCR reviews are carried out for both academic and 
administrative/support units, and since their functions differ, the emphasis in 
both the Self Study Report and the Unit Review Committee Report (URC 
Report) are also different. The emphasis in the case of academic units is on 
the teaching and research function, whereas in the case of 
administrative/support units the emphasis is heavily managerial. In the 
beginning there was considerable discussion about the inclusion of 
administrative/support units in the PACCR process. Reviews of such units 
tend to be carried out only if there is a crisis. The decision to routinely 
include administrative/support units was a wise one, and the process has 
worked well. What we have heard about the performance of our 
administrative/support units is generally favourable. The University is served 
by a dedicated staff trying to do a good job. This does not mean that there 
are not many recommendations about needed improvements. Better 
communication is needed, both internal and external. This matter is almost 
invariably raised in the reviews of administrative/support units, and 
frequently also in reviews of academic units. 


It is probably accurate to say that, in the early years of the PACCR 
process, departments judged the success of a review partly by the increased 
resources they managed to get, usually in the form of new positions. We 
have noticed a significant change in the last two or three years. As a result of 
the difficult financial situation the University faces, the expectations of 
departments are not what they used to be, although the President is still able 
to provide some help when a Unit Review Committee has pointed to an 
urgent situation that needs immediate attention. What we have noticed is that 
departments are now much more aware that a PACCR review is part of a 
long-range process and that, properly used, the URC Report can play a major 
role in the program improvement. 


In a recent letter a chair writes that the URC Report “will be a great help to 
us in the next few years,” and that it “will do much to guide our thinking in the 
future.” 


Mr. Brian Silzer, who became Registrar just after the completion of the 
review of the Registrar’s Office, has told us that he used the Unit Review 
Committee recommendations as an agenda for what needed to be done and 
that most of the recommendations have been enacted. Most important was 
the Unit Review Committee’s analysis of the problems the University faced 
with respect to registration procedures. This analysis encouraged the 
Registrar's Office to pursue the introduction of telephone registration more 
vigorously than might have been done otherwise and to introduce the new 
registration system that has been widely praised. 
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The following excerpt from the Department of English Response to the 
report of the Unit Review Committee will provide a graphic example of what 
can happen when a department fully enters into the process set in motion by 
an URC Report. The excerpt is here quoted with permission of the Chair of 
English. 


On October 16 and 17, 1986, acting on a suggestion in the URC Report, the 
English Department (having cancelled all classes for the two days) went on a 
Retreat to discuss the matters raised in the URC Report and any other issues of 
importance to the Department's life. 73 people attended the Retreat the first day, 
68 the second day; these included nearly all the professors not on leave, 12 
full-time and two part-time sessional lecturers, ten elected graduate students, 
two elected Honours students, three non-academic staff, and the Department's 
APO. The Retreat marked the culmination of intensive meetings by the 
Department, including one pre-Retreat meeting of the Department Council and 
meetings of all eighteen study groups set up in response to the URC Report. 
Department members were assigned to three groups each, by lot, and each group 
chose its own speaker; some groups, whose topic required specialized knowledge, 
were assigned a resource person who had experience in the area. All groups met 
at least once before the Retreat, and some met several times; all groups met 
again during the Retreat. The Retreat itself comprised small-group sessions 
alternating with plenary sessions at which group speakers reported and the 
Department as a whole discussed the issues raised by the groups and voted on a 
total of 79 specific proposals. The Retreat was exhausting but very valuable; by 
the end of it many participants wished we could some day have a less structured, 
less legislative Retreat, but most agreed that the backlog of issues overdue for 
examination left us little choice but to proceed in this grueling manner this year. 
After the Retreat, committees immediately began the work referred to them at the 
Retreat. Chairman's Advisory Council met twice to discuss the Retreat aftermath 
and formulate a response to the URC Report based on the Retreat and its 
attendant meetings; and after a response had been drafted, the whole English 
Department Council met to discuss the response and suggest improvements. 
Thus we can report that the deficiency of Department participation in the 
Self-Study that worried the Unit Review Committee has, during later stages of 
the Review, been amply remedied. Several Department members believe they 
have noticed an improved morale among us since we went through this process 
together; and one symptom of that improved morale may be that when the 
agenda was announced for a Chairman's Advisory Council meeting to staff two 
new committees created at the Retreat, three people volunteered to sit on those 
committees—an unprecedented occurrence. 


The Department of English is by no means the only academic unit that 
has used a formal retreat to discuss in depth the issues raised by the Unit 
Review Committee. Where it has happened, we have had reports that are like 
the excerpt quoted. In virtually every case an URC Report is the occasion for 
long and thorough discussions in a given department. Important, positive 
changes often result from these discussions. In a small number of cases the 
Response has been perfunctory, and we must conclude that the unit in 
question is not taking the review seriously. 
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The majority of departments, however, consider the review process an 
opportunity to examine the past, evaluate the current situation, and consider 
the direction for the future. Also clear is that, without an URC Report, 
comprehensive discussions would not likely take place in departments. At the 
same time, many of the issues raised by Unit Review Committees and later 
addressed by departments are complex and cannot be settled by simple 
motions, however well intentioned. Some of the problems raised will take a 
long time to be worked out; hence we say that a simple balance sheet cannot 
be produced. 


One result of PACCR reviews has been a greater awareness of problems 
within a given unit, leading in many cases to an enhanced understanding, 
not only among the members of a unit, but also among and across units 
within the University. The increased interaction that is often the result of a 
PACCR review is an important benefit. And we should add that, because 
many of the reviewers as well as the people who provide input for a review 
come from outside the University, reviews have created a greater awareness 
of the activities of the university by outside groups both within and outside 
the Province.* 


Reviews have often brought important matters to a higher level of 
visibility and forced an examination of matters that had previously been 
ignored. Reviews have enabled people in decision making roles to obtain a 
fresh point of view about the strengths and weaknesses of a unit. Information 
that is useful for planning is often made available, and in theory at least, the 
decisions made should be of higher quality. 


Because questions and claims about teaching workloads have repeatedly 
come up, PACCR felt compelled to examine the nature of the teaching 
workload in detail and report the findings to the University community (see 
Section III). We concluded that, for sound comparisons of teaching 
workloads, the number of lectures, class sizes, scheduled nonlecture 
activities, and teaching assistant contributions must all be properly taken into 
consideration. The weighting to be assigned to these factors is a matter for 
sound informed judgment if equitability is to be obtained when making 
teaching-load comparisons among diverse departments. (see Section III and 
also the Sixth Annual Report, pages 12-22 for details.) 


PACCR has also had to address the relation between teaching 
effectiveness and research productivity, particularly in light of claims in 
various self studies that teaching loads and other activities in a given 
department are so onerous as to preclude much in the way of research and 
scholarly work. Although objective information on this subject is difficult to 
obtain, PACCR has attempted to draw some conclusions from the information 
we were able to gather. When a large group of staff is considered, the major 
conclusion is that the overall correlation coefficient for the group relating 
undergraduate teaching competence to research productivity is near zero. 


* The following is from a letter from one of our reviewers. “I want to thank you for the 
opportunity to be part of this committee. I enjoyed my association with the other committee 
members and the University of Alberta people we came in contact with.” One who was asked to 
provide input writes, “Please convey my appreciation to PACCR for inviting me and other . . . 
representatives to the Unit Review. The University should be congratulated on its process to 
invite input from the practicing field in planning for ongoing program improvement.” 
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The broader question of overall workload has had to be addressed. In 
previous annual reports, especially the Sixth, PACCR has drawn attention to 
aspects of total workload in a preliminary way. The general components of 
workload have been reviewed, and discussed in relation to teaching 
workloads, though a more penetrating analysis remains to be done. 


It is often said that the self study is the most important document 
produced in the PACCR process. A number of chairs/directors have told us 
that if nothing beyond the self study happened, the PACCR process would 
be worthwhile*. That observation is true, up to a point. Certainly, many 
important decisions are made by departments during the deliberations that 
take place in the preparation of the Self Study Report. Most self studies are 
thorough examinations of the history and the current state of an academic or 
administrative/support unit, with some attention given to future 
developments. A few of these documents have been truly outstanding, and 
some have been weak. 


But whatever the quality of the self study, the effort required to produce 
it would not be made without the awareness that a committee of eminent 
specialists in the discipline under review, together with trusted colleagues 
from our own University, were going to examine the self study and the 
department. 


Many people regard the Unit Review Committee Report as the crucial 
document in the PACCR process. A typical URC Report will have many 
recommendations, sometimes up to forty, ranging from weighty to relatively 
trivial matters. Some of the recommendations can be quickly and easily dealt 
with. When, for instance, a department agrees with suggestions as to some 
changes in the undergraduate curriculum, or ways in which relations with its 
graduate students can be improved, these changes can be introduced fairly 
quickly. In such cases the benefits of a review are immediately apparent. On 
the other hand, some recommendations take time for implementation, and 
the benefits are apparent only much later. For example, in a number of 
academic units there have been recommendations that more rigorous 
promotion standards be used, and in nearly every administrative/support unit 
the point has been made that better communications with users is important. 


Unit Review Committees often recommend profound, and sometimes 
radical changes in the work of a department. Such recommendations cause a 


*The following excerpt from the summary of the Faculty of Extension self study might be 
informative: “The Self-Study Report is a snapshot of the Faculty of Extension on March 31, 1987. 
It outlines the creative past of an organization with vision, and the energy and commitment to 
move forward, even in times of adversity. It describes the present--the Faculty today--a diverse, 
complex operation; its finger on the pulse of the community, responding to educational needs as 
they arise. And finally, the report presents a vision of the Faculty’s future role--a role not unlike 
that of its past, since the requirement for educational programs and materials to meet the lifelong 
learning needs of Albertans continues. The ways in which that role will be fulfilled will change, 
due to what would seem fundamental economic and social changes within western society and 
the world community. It will also be affected by changes within the University, as it determines 
its role for the future. 

The self-study process provided the Faculty with,a framework--one which has supplied the 
means to identify its organizational strengths and limitations; and to reflect on a wide range of 
issues, from opportunities for improvement of administrative policies and procedures, to the 
impact of environmental trends on educational needs, and their implications for the Faculty. 
Preparation of the Self-Study Report required that the Faculty take an objective look at itself as an 
organization: the result being that the areas requiring a decision by the Faculty are becoming 
clear, as are the means to make these decisions and initiate action.” 
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great deal of debate in departments; even when accepted, they cannot be 
easily or quickly implemented. Let us give some examples: 


1) Criticism of existing promotion policies. 


2) Comments about low scholarly productivity, and suggestions about 
how such a situation can be improved. (Both promotions and 
productivity comments have come up many times.) 


3) Recommendations about how retiring staff members should be 
replaced so as to bring about both a revitalization of a department 
and a reordering of scientific and scholarly priorities. 


4) Suggestions about what a search committee should look for when 
new staff members are to be appointed. 


5) Recommendations about a radical reexamination of the 
undergraduate, and particularly the graduate, curriculum. 


6) Recommendations concerning the supervision of graduate students. 


7) Recommendations that certain areas of graduate work be 
discontinued unless a department can make sufficient resources 
available to make the program viable. 


8) Suggestions about future developments in a department’s work 
designed to make the department a leader in its field. 


9) Recommendations designed to eliminate duplication of effort. 


10) Suggestions concerning the improvement of communications 
between a departmental administration, both academic and 
administrative/support, and members of the staff. Recommendation 
that administrative/support units need to strive for more 
consultations and better communication with their clients. 


11) Recommendations that administrative/support units need to do a 
better job of acquainting the campus community about the services 
they have to offer. 


Although this is by no means an exhaustive list of the kind of 
recommendations, criticisms, and suggestions to be found in URC Reports, it 
will give some idea of their wide-ranging nature. It will also be apparent that 
the implementation of such recommendations may take a long time, and 
whatever benefits may eventually derive from the process cannot be easily 
tabulated. The process can act as a catalyst for the examination of matters 
frequently ignored. 


Some departments are of course always reviewing their programs and 
making changes. Nevertheless, the PACCR process undoubtedly encourages 
serious and systematic consideration of the total work of a department and 
acts as a catalyst in program improvement, which as explained in Section I is 
the main aim of the review process. There is no doubt also that without the 
impetus of the PACCR process many changes that have taken place would 
not have occurred. Many serious problems that had been allowed to fester 
have been resolutely dealt with after strong recommendations by Unit Review 
Committees. 


What is the collective profile of the University of Alberta that has 
emerged after seven years of PACCR reviews? Some generalizations can be 
attempted. We can say that the University of Alberta is a first-rate university, 
particularly in its undergraduate programs. These programs have been 
generally praised, and the strong emphasis of the University on 
undergraduate teaching has been lauded by virtually every Unit Review 
Committee. 
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The reputation of the University as a graduate and research institution is 
more mixed, especially when that reputation is compared with that of top 
international institutions. A few departments (and these can be found spread 
throughout all the major faculties) have achieved truly international standing. 
The members of their staff are internationally known, and the quality of their 
graduate students is recognized by other institutions who seek them out for 
appointments. Just below this select group are a number of departments on 
the threshold of achieving international standing. A still larger group has a 
national reputation. The others must be rated as important on the provincial 
scene, but have yet to create national or international reputations. 


The last statement is not meant to be pejorative. The primary 
responsibility of the University of Alberta is of course to the citizens of this 
province; some of our departments fulfill that function in their teaching, but 
are weak in research in general or in the kind of research that would become 
known beyond the province. Several departments restrict their research to 
purely provincial matters, usually in the form of contract research 
commissioned by government departments or by local industry. While such 
research is important, Unit Review Committees have often commented that it 
does not usually deal with really fundamental matters, and they have urged 
such departments to broaden their research activities. 


It should be clear to everybody that an academic department earns 
national and international reputations through the achievements of its staff 
and the quality of its graduate students. Scholarly, scientific, and creative 
activity must therefore be emphasised, and departments must attempt to 
attract top graduate students. It is now, we believe, widely accepted that 
teaching alone, however effective, cannot be the sole criterion for promotion 
to the rank of full professor. It is gratifying for us to report that some 
important steps have been taken to address these matters. 


There has also been some improvement in our standing in national 
competitions of the major granting agencies. The Faculty of Graduate Studies 
and Research has taken steps, such as the creation of the Andrew Stewart 
Fellowships, designed to recognize and reward excellence in graduate student 
research. Measures such as these must ultimately strengthen the reputation of 
the University. But more needs to be done. For instance, the University as an 
institution has taken a somewhat cavalier attitude to the placement of 
graduate students. Many leading universities, both in Canada and the United 
States, have been much more aggressive in this respect, not leaving things to 
chance or to the vagaries of the academic market place. Although our best 
departments have been active in promoting the interests of their graduate 
students, too many have taken a passive attitude. There is much room for 
improvement in this respect. 


Some superficial attitudes unfortunately still exist on our campus. A 
member of one department, strongly criticized by the Unit Review Committee 
for the poor quality of its graduate program and for its lax standards in 
recruiting good graduate students, replied that it is hard to attract good 
students because the winters in Edmonton are very cold! A member of 
another department similarly criticized responded by saying that he didn’t see 
why good students should come here when they can go to Harvard or to 
exciting cities like London and Paris! We hardly need to comment, except to 
say that we can imagine the derisive snorts of contempt that would have 
come from the late Art McCalla had such comments been made in his 
presence. 
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The University of Alberta is considering a major new initiative. Part of 
this initiative envisages the growth of the graduate population to 5000, or 
20% of the total enrollment of the University. we must recognize that this 
expansion of the graduate population would not be simply a numerical 
expansion. The whole nature of the University would be profoundly affected. 


The PACCR process has already helped enormously in identifying major 
problems and in pointing the way to their solutions. We do not wish to 
pretend, however, that the results of reviews have been uniformly 
satisfactory. Much depends on the attitudes of the people in departments. 
When there is no will to bring about important changes, then little will 
happen. When there is such a will, particularly within the leadership of a 
unit, then much can and does happen. 


We have said earlier in this report that some recommendations by Unit 
Review Committees, even when these are accepted by a given unit, will take 
a long time to implement. Sometimes the impetus to bring about change, 
initiated by the review process, becomes weakened over time or is abandoned 
altogether. We are aware of cases where major recommendations, even 
though accepted by a department, either academic or administrative/support, 
are in fact quietly shelved, and no effort to implement them is made. It is 
true that the Vice-President (Academic) or the Vice-President (Administration) 
makes inquiries, but we are not sure that this process by itself is enough to 
keep the momentum for improvement alive. We believe therefore that the 
deans for the academic units and associate vice-presidents for 
administrative/support units should play a more active role in the closure 
phase than they have hitherto been assigned. 
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